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ABSTRACT 



Naval Aviation has been challenged to cut its 1996 human factors related Class A 
flight mishap rate in half by the year 2000. Investigations show that human caused flight 
mishaps have not declined as rapidly as mechanical ones. From fiscal year 1990 through 
1997, maintenance was a causal factor in 17 percent of Class A flight mishaps. Presently, 
there is an ongoing effort to identify factors contributing to human error in aviation 
maintenance. One major component is the development of an instrument to assess safety 
climate and posture in maintenance operations. This thesis is the climate safety 
assessment portion of this effort. It utilizes and adapts an existing Model of 
Organizational Safety Effectiveness (MOSE) to achieve an understanding of the possible 
influences of organizational factors on aviation maintenance. This thesis develops and 
administers a prototype Maintenance Climate Assessment Survey (MCAS) that provides 
a tool for assessing safety in maintenance operations. The study has 268 participants 
from three Reserve squadrons that represent the spectrum of aviation communities. The 
prototype MCAS is comprised of 67 questions developed from 155 candidate questions. 
Each question uses a Likert type rating scale, which allows participants to express 
opinions for each item presented. Cluster and Factor analysis is used to identify 
redundancies between items and how items clustered according to the MOSE 
components. The product of this study is a finalized MCAS with 35 questions that can be 
used by the Squadron command and Aviation Safety Officer to assess their unit’s safety 
posture in conducting scheduled/unscheduled maintenance operations. 
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EXECUTIVE SUMMARY 



Naval Aviation has been challenged to cut its 1996 human factors related Class A 
flight mishap rate in half by the year 2000. Numerous engineering and programmatic 
intervention strategies have been put in place to reduce aviation mishaps, and 
subsequently, the annual flight mishap rate. Improved aircraft reliability, advanced 
cockpit technologies, upgrades to avionics, and new acquisitions have lead to 
improvements that have helped to lower the mishap rate. However, mishaps are still 
occurring. Investigations show that while the mechanical-caused flight mishap rate has 
fallen significantly to an all time low, the human-caused flight mishap rate has not 
declined as rapidly and has possibly increased in the last 10 years (Naval Safety Center, 
1997). 

On the heels of the 29 January 1996 Nashville, Tennessee, F-14 fighter jet. mishap 
that killed the aircrew and several local residents. Vice Admiral Bennitt, Commander 
Naval Air Force, U.S. Pacific Fleet, created a Human Factors Quality Management Board 
(HFQMB) to improve aviation safety by preventing human error. The HFQMB was 
chartered to examine factors contributing to human error, especially aircrew error. 

In 1997, the Navy and Marine Corps had an impressive year; their combined class 
A flight mishap rate was the second lowest ever recorded and it was the Navy’s best ever 
recorded (NSC, 1998). Given this success, a shift to improve maintenance was initiated. 
Of all the identified causal factors that contributed to Class A flight mishaps from fiscal 
year 1990 through 1997, 17 percent involved maintenance. “Maintenance” causal factors 
are primarily due to human errors by maintenance personnel and/or maintenance 
supervisors. The maintenance thrust of the HFQMB adapted the same approach of 
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benchmarking, mishap data analysis, and climate safety assessment. This thesis is the 
climate safety assessment portion of an ongoing study of Naval Aviation maintenance 
operations. 

Organizational culture has an effect on the entire organization. This is especially 
true in high-risk organizations where potential outcomes of a system failure can be 
catastrophic and costly. Naval Aviation is recognized as a high-risk organization, which 
can experience devastation if something goes awry. This also holds true for aviation 
maintenance as well as operations. This study involves the analysis of data from a 
prototype survey that is based on an existing model of high-risk organizations. The 
purpose of this study is to develop, administer, and validate a Maintenance Climate 
Assessment Survey (MCAS) to assess the effectiveness of Naval Aviation maintenance 
operations in the management of risk. It utilizes and adapts an existing Model of 
Organizational Safety Effectiveness (MOSE). This study analyzes the MCAS data; the 
results are intended to improve our understanding of the possible influences of human 
factors in aviation maintenance related mishaps, and to provide a tool that can be used by 
the Squadron command and Aviation Safety Officer to assess their unit’s safety posture 
in conducting scheduled / unscheduled maintenance operations. 

The study includes a total of 268 participants, of whom 212 are maintainers, from 
three Reserve squadrons that represent a variety of Naval Aviation communities. The 
prototype MCAS comprised 67 questions developed from 155 candidate questions. Each 
question contained five point Likert scales (strongly disagree, disagree, neutral, agree, 
and strongly agree) which allowed participants to rate items according to their views. 
The questions fit into six components of the MOSE: Process Auditing, Reward System 
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and Safety Culture, Quality, Risk Management, Command and Control, and 
Communication / Relationships. The data analysis entails descriptive statistics, cluster 
analysis and factor analysis for objective identifications of redundancies among items and 
how items clustered according to the MOSE components. Redundant questions are 
eliminated and questions with common intent are combined. Some questions remain the 
same. Thorough analysis of the questions produce a finalized MCAS with a total of 35 
questions that can be used for individual squadron self-assessment of safety posture in 
scheduled / unscheduled maintenance operations. 
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I. 



INTRODUCTION 



Naval Aviation is an inherently complicated system, from flying the platform into a 
hostile environment to landing it on an aircraft carrier. Complex systems have many 
forces acting on them, some synergistically, some separately, all having either positive or 
negative effects (Perrow, 1986). Such forces and the effects they have on the system are 
difficult to grasp and understand (Bond, Byran, Rigney, & Warren, 1975). These forces 
can lead to pilot errors, cause failures in the platform or create a range of causal factors 
that can lead to the loss of a platform, or worse, lives, thus having a direct effect on 
combat readiness. Warfighting readiness and victory in combat can be achieved by 
keeping the platforms functional and the pilots alive (Nutwell & Sherman, 1997). 

A. BACKGROUND 

The Naval Aviation Safety Program (Chief of Naval Operations Instruction 
3750. 6Q, 1989) was established to deal with aviation mishaps and any related causal 
factors. The purpose of this program is to preserve human and material resources while 
its objective is to eliminate hazards, the causes of damage and injury. In essence, the 
program was set up to determine all hazards that can become causal factors leading to 
aviation mishaps. 

Numerous efforts have been employed in attempts to lower the Class A flight 
mishap rate (Figure 1). These efforts resulted in the creation of several institutions and 
programs, including the Naval Aviation Safety Center (now Naval Safety Center - NSC), 
the Naval Aviation Maintenance Program (NAMP), the Fleet Replacement Squadron 
(FRS) concept, the Naval Air Training & Operating Procedures & Standards (NATOPS), 
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and the Squadron Safety Program (SSP). Additional efforts include Aircrew Coordination 
Training (ACT), Human Factors Councils/Boards (HFC/B), and Operational Risk 
Management (ORM). 




Figure 1. FY 50-96 Flight Mishap Rates and Intervention Strategies. 
From “Naval Safety Center Brief,” by Admiral F. Dirren, April 1997. 



All of the above efforts are intervention strategies put in place to reduce aviation 
mishaps and the annual flight mishap rate. The First Five programs mentioned are design 
and programmatic in type. The last three break from an engineering and management 
focus to concentrate more on human factors issues. Improved aircraft reliability, 
advanced cockpit technologies, upgrades to avionics, and new acquisitions have lead to 
improvements that lower the mishap rate. However, mishaps are still occurring. Since 
1990, approximately 4 of every 5 Department of the Navy (DON) Class A flight mishaps 
(those involving aircraft damage in excess of one million dollars or fatal injury) involve 
human factors (Department of the Navy (DON), 1997). For example, in 1997, of the 27 
aircraft destroyed, 23 involved human factors. 
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Naval Aviation has been challenged to cut its 1996 human factor related Class A 
flight mishap rate in half by the year 2000. To do this, it must systematically and 
continuously improve all of the processes, programs, and systems affecting the safety of 
Naval Aviation operations (Nutwell & Sherman, 1997). The Naval Aviation Class A 
flight mishap rate has been significantly reduced over the past few decades. However, it 
appears to have reached a plateau over the past decade as indicated in Figure 2. 

4 
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88 89 90 91 92 93 94 95 96 97 
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Figure 2. Naval Aviation Class A Flight Mishap Trend FY 88-97. 

Rate is Defined as the Number of Class A, B, & C Flight Mishaps Per 100,000 Flight Hours. 

Though the Navy and Marine Corps flight mishap rate has generally decreased 
over the past two decades, the rate of decline has slowed. Investigations show that while 
the mechanical-caused flight mishap rate has fallen significantly to an all time low, the 
human-caused flight mishap rate has not declined as rapidly and is possibly leveling off 
or even increasing in recent years (NSC, 1997). Some conclude that this split is due to 
the complexity of the aircraft and current pressures of operations tempo, personnel 
downsizing, etc (Schmidt, 1998). Figure 3 shows this relationship. 
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Figure 3. Mechanical/Catastrophic Failures vs Human Errors Over All Mishaps. 

From “Naval Safety Center Brief,” by Admiral F. Dirren, April 1997. 

Figure 4 shows the percentage of determined causal factors that contributed to 
Class A flight mishaps from FY90 - FY97. Supervisory factors were present in 62 
percent of these flight mishaps. Examples of supervisory factors include inadequate 
command supervision of flight operations or safety, failures to correctly assess mission 
risks, and the inappropriate handling of a known high-risk situation. The aircrew was 
found to be a causal factor in 56 percent of the mishaps. Material factors, defined as 
material failure of the component, were present in 39 percent of the mishaps. 
Maintenance was found to be a causal factor in 17 percent of mishaps. This 
“maintenance” causal factor was primarily due to human errors by maintenance personnel 
or maintenance supervisors. Facilities factors, relating to human errors in conjunction 
with airfield operations, contributed to five percent of the mishaps. 
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Figure 4. Navy and Marine Corps Fiscal Year 90-96 Class A Flight Mishap Causal Factors. 

From “Naval Safety Center Brief,” by Admiral F. Dirren, April 1997. 

On the heels of the 29 January 1996 Nashville, Tennessee, F-14 fighter jet mishap 
that killed the aircrew and several local residents, Vice Admiral Bennitt, Commander 
Naval Air Force, U.S. Pacific Fleet, created a Human Factors Quality Management Board 
(HFQMB) to improve aviation safety by preventing human error. The charter of the 
HFQMB is to analyze and improve each of the processes, programs, and systems that 
impact human performance in aviation with the purpose of dramatically reducing the 
annual flight mishap rate (DON, 1997). In other words, its main objective is to analyze 
human factors involvement in past Naval Aviation mishaps and in present Naval Aviation 
operations. Preventing a mishap contributes to readiness and mission success by keeping 
people and airplanes available through controlling safety-related hazards. Most of the 
HFQMB initiatives directly contribute to mission readiness, as well as indirectly, through 
mishap prevention (Nutwell & Sherman, 1997). 
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Three processes define the HFQMB methodology: Benchmarking, Mishap Data- 
Analysis (MDA), and Command Safety Assessment (CSA). These lead to brainstorming 
and potential intervention strategies, which in turn result in recommendations presented 
to the Navy’s Air Board of Flag officers (Figure 5). Benchmarking occurred with visits 
to commercial airlines, private industry, sister services, NASA, the FAA, NATO partners, 
and academia. The objective was to examine the culture, training, safety, and leadership 
of each organization; to identify and leam about successful safety practices in order to 
adopt best practices, identify lessons learned, and avoid “re-inventing the wheel.” 




Figure 5. HFQMB Methodology. 

From “AIRPAC Brief,” by CDR J. Schmidt, March 1998. 

The MDA categorizes the human error elements of a mishap into three groups: 
unsafe supervisory conditions, unsafe aircrew conditions, and unsafe aircrew acts. The 
first two are seen as latent conditions and the last as active failures. Unsafe supervisory 
conditions include planned inappropriate operations, failure to correct known problems, 
supervisory violations, and inadequate supervision. Unsafe aircrew conditions pertain to 
aeromedical, personnel readiness and cockpit resource management (CRM) problems. 
Unsafe aircrew acts pertain to slips, lapses, mistakes, routine violations, and exceptional 
acts of the aircrew. Class A flight mishap data analysis from fiscal year 1990 through 
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1996 showed the following. Supervisory failure is a causal factor in 62 percent of Class 
A flight mishaps during fiscal year 1990 through 1996. Fifty one percent of the 
supervisory failures are labeled as unsafe supervisory conditions. The unsafe aircrew 
events contained: 28 percent of the flight mishaps as aeromedical, 48 percent as CRM 
and three percent as readiness. The unsafe aircrew act event contained 87 percent as 
accidental and 40 percent as violations. (Nutwell & Sherman, 1997). Notably, in a report 
issued in 1997 by the DON, forty percent of the volations were conscious aircrew 
breeches of procedures. Standard Operating Procedures (SOP) and NATOPS. 

Ciavarelli’s and Figlock’s (1997) CSA study was conducted to assess a 
command’s safety climate from an aircrew perspective and the perceived effectiveness of 
its safety related programs. Its objective was to analyze the effectiveness of managing 
risks in flight operations. The study, based on a model of high reliability organizations, 
shows significant room for improvement in risk management, human factors evaluation, 
and aircrew coordination training. The study’s survey develops a tool that provides 
commanding officers and aviation safety officers (ASO) a capability for self assessment 
of the effectiveness of their commands safe operations (Nutwell & Sherman, 1997). 

The HFQMB formed ten Process Action Teams (PAT) for areas that require 
intervention: Leadership, Policy, Organizational Effectiveness, Safety Information 
Management, Training and Qualifications, Standard Operating Procedures, Mishap 
Investigations, Operational Risk Management, Human Factors Evaluation and 
Aircraft/Aircrew Systems. The PAT were tasked to assess each area using the 
information taken from benchmarking, the MDA, and CSA, to identify area 
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improvements, and recommend actions resulting in an awareness of human factors, causal 
factors of mishaps and, ultimately, to achieve a mishap rate reduction. 

To date, there has been little emphasis on the potential contribution of human 
factors on maintenance operations. The HFQMB’s first thrust was aimed at supervisory 
(62 percent) and aircrew (56 percent) causal factors due to the large percentage of 
mishaps involved, whereas discussed earlier, maintenance was found in 17 percent of the 
mishaps. In 1997, the Navy and Marine Corps had an impressive year; its class A flight 
mishap rate was the second lowest ever recorded and it was the Navy’s best ever recorded 
(NSC, 1998). Success in the intervention of these two areas has led to interest in 
developing efforts directed at the maintenance causal factors. 

The same three-pronged approach was adopted by the HFQMB for taking an in- 
depth look into Naval Aviation maintenance operations for the identification of hazards 
and the development of intervention possibilities (Figure 5) (Schmidt, 1998). 
Benchmarking is currently underway and mishap data analysis is now completed. The 
benchmarking efforts have determined that risk management contributes to enhanced 
safety in terms of maintenance error prevention, lower mishaps, and reduced personal 
injuries. Organizational culture and climate, facilitation of crew coordination and team 
performance, effective training and certificate programs, and emphasis on policy and 
procedure adherence are all areas where risk management contributes to enhanced safety. 
These findings reflect some of the problem areas found in an analysis of Class A flight 
mishaps that involved maintenance depicted issues related to supervisory, crew 
coordination, training/qualifications, and adherence to policy and procedures. The 
results from benchmarking and mishap data analysis in conjunction with the previous 
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CSA questionnaire were used to develop a candidate list of questions for a maintenance 
questionnaire. 

B. PURPOSE 

The purpose of this study is to develop, administer, and validate a questionnaire 
survey to assess the effectiveness of Naval Aviation maintenance operations in the 
management of risk. This study analyzes survey data and builds on a Model of 
Organizational Safety Effectiveness (MOSE) developed by Dr. Ciavarelli and LtCol 
Figlock (1996) of the School of Aviation Safety, Naval Postgraduate School, Monterey, 
CA. The results will improve our understanding of the possible influences of human 
factors in aviation maintenance related mishaps, and will provide a tool for assessing 
safety in maintenance operations. The intent is to identify potential interventions in an 
effort to lower human error in maintenance-related mishaps and the overall mishap rate. 

C. PROBLEM STATEMENT 

Given the sustained presence of human error in aviation maintenance, it is 
essential to develop a valid Maintenance Climate Assessment Survey targeting the 
maintainer’s perspective on safety and flight practices. Given this need, the existing CSA 
questionnaire based on the present MOSE must be adapted to address maintenance issues. 
It entails possible expansion and necessary modification of the existing questionnaire, 
which is geared towards aircrew, and the development of maintenance specific items. 
This thesis explores the following questions: 

1. Can the present MOSE be adapted to capture the maintainer’s perspective? 

2. Do the close-ended questions (questions with fixed responses) in the MCAS 
conform to the existing MOSE components? 
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3. Can the prototype MCAS be reduced through either the elimination of 
redundant questions or combination of such questions? 

4. Are there discemable differences between squadrons surveyed, and does there 
appear to be any consensus? 

5. Does the maintenance data support the components / structure of the 
maintenance adapted MOSE? 

D. SCOPE AND LIMITATIONS 

The intent of this study is to gain a better understanding of the Naval Aviation 
maintenance safety climate, understanding that efforts are underway to look into the 
supervisory and aircrew factors. Three California based Reserve Squadrons are used as 
subjects in the survey due to their availability, accessibility, close proximity; this effort is 
a preliminary study. Neither Marine Corps units nor active duty personnel participated in 
this study, nor did any Tactical Air Squadrons. Only the Naval Aviation maintenance 
personnel responses are used in this analysis. Incomplete surveys are omitted. 
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II. LITERATURE REVIEW 



A. BACKGROUND 

It has been evident for most of recorded history that people are prone to commit 
errors- “To err is human.” As our technology expands, as our wars multiply, and as we 
invade more and more of nature, we create systems - organizations, and the organization 
of organizations - that increase the risk for the operators, passengers, innocent bystanders, 
and for future generations. Most of these risky enterprises have catastrophic potential, the 
ability to take the lives of hundreds of people in one blow, or to shorten or cripple the 
lives of thousands or millions more (Perrow, 1984). Catastrophic errors in a system are 
often the result of the gradual aggregation of small errors across a system. Considering 
organizations as systems, and as parts of systems, is important in order to understand how 
organizations work, and how to best mitigate risk in them (Grabowski & Roberts, 1996). 
Advances in technology have increased the reliability of most system components, but the 
percentage of human error-related incidents and accidents has remained fairly constant. 

B. STUDY FRAMEWORK 

The theoretical framework for the construction of the maintenance questionnaire 
survey used in this study is derived from the work done by Ciavarelli and Figlock (1997) 
at the School of Aviation Safety, Naval Postgraduate School, Monterey, CA. The 
framework is based on key attributes of successful organizational risk management 
processes identified by Libuser (1994) and Roberts (1990), and from a safety climate 
analysis published by Zohar (1980). The existing CSA questionnaire was generated using 
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academic resources and professional questionnaires, which provided guidance regarding 
design and construction of questionnaires (Ciavarelli and Figlock, 1997). 

Five major areas concerning the effectiveness of organizations at managing risk 
are identified in a model developed by Libuser (1994). These areas included process 
auditing, a reward system, quality of operations, risk perception and command and 
control. Libuser’s work was adapted by Ciavarelli and Figlock (1997) to develop a 
Model of Safety Effectiveness (MOSE) that incorporates standard Naval aviation 
language and safety practices. The five MOSE components are summarized as follows: 

1. Process Auditing - a system of ongoing checks to identify a hazardous 
condition and a process to take corrective actions. 

2. Reward System - a system to recognize and reward safe behaviors, and to 
discourage unsafe behaviors. 

3. Quality standards - Sets control procedures to monitor quality and correct 
deviations to referenced standards. 

4. Risk Management - Systematic process to identify and manage risk. 

5. Command and Control - Established policies and procedures to conduct safe 
operations. 

A complete description of the MOSE components is provided in Appendix A. 

Ciavarelli and Figlock’s (1997) short term goal is to provide the QMB with 
feedback on issues concerning aviation command climate, morale of Naval Aviation 
personnel, squadron workload and resource availability, the estimated success of ongoing 
safety interventions, and other factors related to safely managing Naval flight operations. 
Their long-term goal is to develop, validate and apply methods for assessing and 
mitigating risk in organizations that must, by the nature of their business, conduct 
hazardous operations. They develop and apply questionnaire survey methodology to 
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assess the effectiveness of Naval Aviation units in the management of risks associated 
with flight operations. This study is the first of several planned investigations intended to 
improve the understanding about possible influences of organizational factors in aircraft 
accidents, specifically understanding what possible influences a Naval Aviation 
command may have on the chain of events leading to an aircraft mishap (Ciavarelli & 
Figlock, 1997). 

C. ORGANIZATIONAL CULTURE AND CLIMATE 

Roberts (1990) examines organizations that are successful in managing risks 
associated with hazardous operations. She labels this type of an organization as a “high- 
reliability” organization (HRO), which operates in hazardous environments yet produces 
a very low rate of accidents and incidents. Grabowski and Roberts (1996) contend that 
strong cultures and norms that reinforce their mission and goals characterize HROs. A 
culture is defined as the shared values, beliefs and assumptions that may govern decision 
making, attitudes, safety, and proper conduct within an organization. Norms are rules and 
regulations that guide and control the behavior of individuals and organizations. They 
focus attention on procedures, policies, and reward structures that are consistent with the 
organization’s mission. Grabowski and Roberts (1996) find that various cultures 
operating in large-scale systems are the glues tying the segments together. 

Zohar (1980) finds strong evidence that organizations with successful safety 

programs have a strong management commitment to safety. She determines that in low 

accident companies, top management is personally involved in safety activities on a 

routine basis, and safety matters are given high priority in company meetings and in 

production scheduling. This underlying safety climate refers to the shared perception of 
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an organization’s members that the organization’s leaders are genuinely committed to 
safety of operations. She contends that such a climate results in increased performance 
reliability of workers, good housekeeping, and high design and maintenance standards for 
work environments. 

D. HRO: CLIMATE AND CULTURE 

To operate a complex system successfully, the human-machine system must be 
supported by an organizational infrastructure of operating concepts, rules, guidelines, and 
documents. In high-risk endeavors such as aircraft operations, it is essential that such 
support be flawless, as the price of deviations can be high. When operating rules are not 
adhered to, or the rules are inadequate for the task at hand, not only will the system’s 
goals be thwarted, but there may also be tragic human and material consequences (Degani 
& Wiener, 1994). Both Roberts (1990) and Libuser (1994) believe that organizations 
operating safely and effectively have certain key characteristics in common. Leadership 
style, sound safety management policies, procedure standardization, adequacy of 
resources and staffing, and a defined system for risk management are such characteristics. 
In large-scale systems, maintaining or developing a culture of safety and reliability can be 
important in mitigating risk. Oversight, checks and balances, and strong cultural norms 
are organizational protections and buffers that reinforce an organization’s goals; they can 
mitigate risk by assuring errors are caught, that appropriate organizational norms are 
developed and reinforced, and that the system improves over time (Grabowski & Roberts, 
1996). 
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E. AVIATION MAINTENANCE AND RAMP ACTIVITIES 

As anyone working in aviation maintenance knows, there have been a number of 

serious, even fatal, accidents over the years that were caused primarily by maintenance 
errors (Reason & Maddox, 1998). Boiled down to its essence, the task of a maintenance 
mechanic is to take off and then replace some of the three to four million removable parts 
on a modem aircraft platform. Much goes on in between, of course, but these basic steps 
of disassembly and re-assembly remain constant features of the work — the latter attracts 
by far the largest number of errors (Reason, 1990). The maintenance area is an unstable 
environment in a number of different ways: from the point of view of human factors it 
does not compromise a well-designed work environment. Problems range from complex 
activity and congestion in a poorly delineated environment to the problems of controlling 
the movement of large objects in space with inadequate information and cues (McDonald 
& Fuller, 1994). It is clear that problems with human factors still exist in aircraft 
maintenance facilities - even those that are built in strict accordance with the prevailing 
architectural standards (Maddox, 1998). Therefore, within maintenance activities, the 
human operator’s role is critical to safe and efficient operations. 

The technology of the ground handling process also poses interesting problems for 
safety. Aircraft are designed to fly; they are not optimally designed for ramp operations. 
The aircraft’s size and location in space relative to the operator invite misperceptions and 
misjudgments of distance and location; the fragile skin and appendages of an aircraft are 
easily damaged and such damage if undetected, can have disastrous consequences in 
flight. The cyclical nature of the ramp operator’s job often requires immediate and 
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demanding increases of both mental and physical performance from a resting and perhaps 
fatigued level (McDonald & Fuller, 1994). 

There are many factors within a maintenance and ramp activity that can be 
associated with failures (Reason & Maddox, 1998). These factors can have an adverse 
effect upon the working practices of those within the maintenance and ramp areas and 
hence, human performance. Factors within the workplace are known as local errors, and 
those that lie within the organization, organizational factors. The organizational factors 
create the local errors. 

One such local factor is the knowledge, skills and experience of maintenance 
personnel; personnel can be unfamiliar with a defect or aircraft type, lack specific training 
or skills and have inappropriate experience to perform proper maintenance. A morale 
factor can exist; people clash in personality, get frustrated, or can be unhappy with the 
work situation. Availability of tools, equipment and parts quality may be another factor. 
Other local factors may include fatigue, the pressures of a high workload, problems with 
shift patterns, and environmental problems. Unclear manuals and procedures, quality of 
safety equipment and training may also be present (Maddox, M, 1998). 

An organizational factor may be organization structures; there can be concerns 
about restructuring and downsizing, ill-defined duties, or too many layers of management 
in the organization. People management, the lack of top level awareness of problems that 
technicians face, and ill-defined career paths, may be an issue. Training, operational 
pressures, planning and scheduling problems and communication within the organization 
can also be examples of organizational factors. These factors are usually present within 
most maintenance facilities, and can directly impact the climate within which the 
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maintenance technician or ground handler must cope. The maintenance and ramp activity 
is a dangerous work environment presenting the risk of death or disabling injury to those 
involved. There is an ever-present potential for unreported ground handling damage to 
aircraft while in the hangar or on the ramp. Therefore, policies and procedures are 
needed to minimize this potential (Maddox, 1998). 

F. PHILSOPHY, POLICY, PROCEDURES AND PRACTICES 

Philosophy, policy, procedures and practices, the four P’s, are recognized as 
important aspects in flight deck operations (Degani & Wiener, 1994). Philosophy deals 
with the over-arching view of how the organization will conduct business, including 
flight operations. An organization’s philosophy is largely influenced by the 
organization’s leaders, and also by its culture. Policies are broad specifications of the 
manner in which management expects operations such as training, flying, and 
maintenance to be performed. An organization’s policy embodies the philosophy. 

Procedures should be designed to be as consistent as possible with the policies, 
which are consistent with the philosophy. Procedures exist to specify what the task is, 
when the task is conducted and by whom, how the task is done, and the sequence of 
actions needed to perform the task. Practices are an extension of and are governed by the 
philosophy, policies and procedures within an organization. A practice is the activity 
actually conducted on the flight deck and it is the ultimate factor that determines the 
quality of a systems output. The goal of management is to promote ‘good’ practices by 
specifying coherent procedures (Degani & Wiener, 1994). Maintenance, like flight deck 
operations, is an activity that relies heavily on the four P’s described above; there is the 
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leadership philosophy, the policies within the maintenance department, the Standard 
Operating Procedures (SOP), and the technician’s or mechanics practices. 

G. ASSESSMENT AND INTERVENTION OF RISK 

Looking at all the components within the MOSE and seeing if these apply to the 
Naval Aviation maintenance community is called its assessment. Do the five components 
of process auditing, reward system and safety culture, quality, risk management, and 
command and control relate to maintenance? Assessing maintenance operations shows 
that indeed, these components are present. There is a need for the auditing of 
maintenance logs and SOP. A strong safety culture is a must; everyone must be safety 
conscious and identify safety problems. Quality control is the heart of maintenance, 
maintenance is definitely a high-risk environment, and leadership must be present to 
ensure the commitment to safety. 

Intervention is the gathering of data from a process, the attempts to fit a model to 
the data, and learning more about the process. A better understanding of the 
organizational factors related to safely managing maintenance operations is needed and 
desired. A survey of the maintenance community, based on the MOSE adapted for 
maintenance, is one method of intervention. Surveys are most appropriate when 
information should come directly from people involved with the process. Surveys 
provide data describing attitudes, values, habits, and background characteristics such as 
age, education and income (Fink & Kosecoff, 1985). Ciavarelli and Figlock’s (1997) 
MOSE can be adopted to incorporate maintenance terms and language, thus developing a 
needed tool for intervention. 



18 



H. SUMMARY 



Organizational culture has an effect on the entire organization. This is especially 
true in high-risk organizations where potential outcomes of a system failure can be 
catastrophic and costly. Naval Aviation is recognized as a high-risk organization, which 
can experience devastation if something goes awry. This is also true in the aviation 
maintenance community. Several approaches have been made to improve the overall 
safety of Naval Aviation: a system safety analysis, an analysis of mishap data and surveys 
to improve the understanding of organizational issues pertaining to operators. If the 
concepts of systems safety and organizational accidents are to be advanced, aviation 
management at all levels must be aware of them (Reason, 1990). There is a need to 
develop a survey to gain a better understanding of the organizational influences on 
maintenance error. In looking at different survey methods, a self-administered survey 
needs to be developed for the Naval Aviation maintenance community. 
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III. METHODOLOGY 



A. RESEARCH APPROACH 

The intent of this study is to develop a Maintenance Climate Assessment Survey 
(MCAS) for the Naval Aviation Maintenance community that captures the maintainer’s 
perception and understanding of the risks within their department and in performing their 
mission. This study involves the analysis of data from a prototype survey that is based on 
an existing model of high-risk organizations. This is done to identify factors that may 
improve safety within aviation maintenance. The data analysis entails descriptive 
statistics, cluster analysis and factor analysis. 

B. DATA COLLECTION 

1. Subjects 

A total of three Naval Air Reserve Force squadrons, two located at NAS North 
Island and one at Moffett FAF, are surveyed. The first is a Fixed Wing Fleet Logistics 
Support (VR) Squadron composed of active duty and selected reserve personnel 
providing seven-day-a-week round the clock, worldwide logistics to support the Navy and 
Marine Regular and Reserve forces. The second consists' of a Fixed Wing Maritime 
Patrol (VP) Squadron with a worldwide theater of operations, composed of Selected 
Reservists and active duty personnel (Note: This squadron is decommissioning).- The 
third squadron consists of a Rotary Wing Combat Support (HCS) Squadron with a 
mission of Search and Rescue, Combat Support via the launch and recovery of targets and 
torpedoes on the Southern California Offshore Range (SCORE). 
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The survey respondents are primarily Navy enlisted personnel who work in 
aviation maintenance departments. Maintenance personnel from the VR Squadron 
located at the Naval Air Station (NAS) North Island, CA were used to test the reliability 
and consistency of the prototype survey. The survey was then administered to two other 
groups of maintenance personnel from the VP and HCS Squadrons. 

2. Instrument 

The prototype MCAS consists of 15 demographic and 67 maintenance related 
questions. Demographic questions included inquiries about rank, community, shift 
worked, total years of service, and total years of aviation maintenance experience. They 
also include inquiries about unit home location, rating, age and current maintenance 
qualifications. The demographic questions preserve anonymity of particpants; names, 
social security numbers, work section, etc are not asked for. The MCAS is constructed 
using an existing air operations oriented Command Safety Assessment survey (CSAS) as 
a basis. 

Questions from the CSAS are re-worded to relate to “Maintainers” with 
maintenance terms. A focus group discussed all 156 candidate questions to reduce 
redundancy and ensure proper phrasing (Appendix B lists the candidate questions). 
Survey questions are fitted to the existing components as outlined in the current MOSE 
and new areas are created to fit any remaining questions. The MOSE components consist 
of Process Auditing, Reward Systems and Safety Culture, Quality, Risk Management, and 
Command & Control. Additional questions, proposed by current maintenance experts, 
are also incorporated into the MCAS. All questions are partitioned according to the 
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MOSE components, and the residual questions are examined. They fit under a sixth 
category peculiar to maintenance vice air operations: Communication / Relationships. 

The MCAS questions are separated into the MOSE components of Process 
Auditing, Reward System and Safety Culture, Quality, Risk Management, Command and 
Control, and Communication / Relationships. Eight of the 67 questions cover Process 
Auditing, 10 cover the Reward System and Safety Culture component, 12 cover Quality 
as well as Risk Management, 14 cover Command & Control and 11 cover the 
Communication / Relationships component. Eight of the 67 questions are worded 
negatively, where a positive response is indicated by a low value. The survey questions 
utilize a five point Likert rating scale with the following verbal anchors: Strongly 
Disagree, Moderately Disagree, Neutral, Moderately Agree, and Strongly Agree (The 
MCAS is included in Appendix C). 

3. Procedure 

The MCAS is distributed in person. All persons taking the MCAS are told what 
the survey is, why it is being given and told that their responses are completely 
anonymous. The Fleet Support Squadron received the MCAS during a safety stand 
down. Groups of 10 to 15 people were administered to until all available personnel had 
taken the survey. Each person was given approximately 15 to 20 minutes or more time if 
needed, to take the survey. The Patrol Squadron MCAS was given one on one going 
from office to office throughout the squadron’s workspaces over a drill weekend. The 
MCAS was handed out in a workspace and the participants were individually told what 
the survey was, why it was needed and that their individual responses would not be 
reported. The workspace MCAS took much longer to administer because participants 
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had a job to do. Some MCAS forms were completed right away, some were collected a 
few hours later. Throughout the workday and the workspaces, the MCAS were collected 
on an individual basis. The Combat Support Squadron personnel took the survey at the 
beginning of a safety brief over a drill weekend. The Squadron personnel were split into 
two groups of 30 to 40 people. Each group was given approximately 15 to 20 minutes or 
more time if needed, to take the survey. The MCAS was collected at the time it was 
completed. For each squadron, each survey was hand numbered for future order. 

C. DATA ANALYSIS 

1. Data Tabulation 

Survey demographics and responses are hand entered into an Excel (Microsoft, 
1996) spreadsheet. The spreadsheet consists of rows of respondents and columns of 
survey questions. The first fifteen columns are for demographics, with another sixty- 
seven columns representing the actual survey questions. Results are coded in the 
spreadsheet by assigning scores of 1,2, 3, 4, or 5 corresponding to the Likert scale of 
Strongly Disagree, Disagree, Neutral, Agree and Strongly Agree, respectively. Questions 
8, 20, 22, 39, 40, 41, and 54 are negative in wording and ask respondents to answer 
opposite to how they respond to the other questions. Survey questionnaire items that had 
no response are left blank and were dealt with by S-PLUS (Mathsoft, 1997) as the data is 
transformed into a SPLUS 4.0 data frame for complete data analysis. 

2. Statistical Analysis 

Basic summary statistics are developed. Descriptive analysis is conducted on the 

data to describe basic and general information about the demographic and question 

results. These results include the distribution of survey participants by rank and service 
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classifications, and the total sample, mean and standard deviation for each of the 67 
survey questions. The S-Plus 4.0 (Mathsoft, 1997) program is used for multivariate 
statistical analyses. Clustering methodologies such as agglomerative nesting and divisive 
analysis utilizing the S-PLUS functions AGNES and DIANA are used to answer the third 
research question with the aid of factor analysis. Factor and principal component 
analyses are performed to answer the second and fifth research questions, utilizing the S- 
PLUS functions FACT ANAL and PRINCOMP. 



25 



IV. RESULTS 



A. DESCRIPTIVE STATISTICS 

Descriptive statistics of survey participants are presented in Table 1. For each 
participating Squadron, Fleet Support, Patrol and Combat Support, there are two 
columns. The column titled “All” is representative of all survey participants for the 
respective squadron. The column with the heading “Maint” pertains to maintenance 
respondents only. Table 1 lists the numbers of officers, E1/E3, E4/E6, and E7 and above 
for each squadron as well as the number of TAR, SELRES and USN designated 
personnel. 





Fleet Support 
Sqaudron 


Patrol 

Sqaudron 


Combat Support 
Sqaudron 


All 

Sqaudrons 


Participants 




All 


Maint 


All 


Maint 


All 


Maint 


All 


Maint 


Officers 


14 


1 


13 


0 


14 


0 


41 


1 


E1/E3 


7 


7 


2 


2 


7 


7 


16 


16 


E4/E6 


77 


68 


71 


67 


42 


39 


190 


174 


E7+ 


10 


10 


7 


7 


4 


4 


21 


21 


Total 


108 


86 


93 


76 


67 


50 


268 


212 




















SELRES 


65 


47 


59 


45 


34 


26 


158 


118 


TAR 


42 


38 


34 


31 


32 


23 


108 


92 


USN 


1 


1 


0 


0 


1 


1 


2 


2 


Total 


108 


86 


93 


76 


67 


50 


268 


212 



Table 1. Distribution of Survey Participants. 



From Table 1, the maintenance personnel involved in taking the survey represent 
79 percent of the total survey population. Maintenance E4/E6 enlisted personnel 
represented a total of 82 percent of the data analyzed in this thesis. This is important to 
note because most of the maintenance work and work supervision is done at this level. 



27 



B. MAINTENANCE RESPONSE STATISTICS 



The MOSE component results for maintenance respondents are presented in 
Tables 2 through 7 (The MOSE component results for all respondents are tabulated in 
Appendix D). Tables 2 through 7 list the maintenance results for all three Squadrons for 
each of the six MOSE components. The tables show total respondents for each question 
(n), average rating for each question on a “1” to “5” Likert scale and the standard 



deviation for each question. It is interesting to note that average results amongst the 
squadrons are relatively similar. Also noted is that for the eight questions of the MCAS 
worded negatively, the respondent’s answers reflect as expected. 

The first MOSE component presented in Table 2 is Process Auditing. There are 
eight questions in this component. The only question that raises a concern is number 59 
(overall mean = 2.73). All three Squadrons rate this question below neutral, in the 
disagree direction of the Likert scale. The question entails the use of safety staff to 
manage personnel at risk. A better, more effective use of safety staff and a higher 
presence of safety staff within the squadron may be a possible way to influence this 
perspective in the positive direction. The remaining eight questions are answered 



positively (mean range = 3.08 to 4.83). 



Question 


Fleet Support 
Squadron 


Patrol 

Squadron 


Combat Support 
Squadron 


Combined 

Squadrons 




Total 


Avg 


StdDev 


Total 


Avg 


StdDev 


Total 


Avg 


StdDev 


Total 


Avg 


StdDev 


1 


86 


4.42 


0.458 


76 


4.38 


0.653 


50 


4.42 


0.758 


212 


4.41 


0.623 


2 


86 


3.79 


0.897 


76 


3.79 


0.838 


50 


3.74 


1.121 


212 


3.77 


0.952 


3 


86 


4.47 


0.699 


75 


4.19 


0.766 


49 


4.33 


0.774 


210 


4.33 


0.746 


4 


86 


4.41 


0.879 


75 


4.29 


0.785 


50 


4.16 


0.889 


211 


4.29 


0.851 


5 


85 


4.60 


0.576 


76 


4.83 


0.444 


50 


4.62 


0.667 


211 


4.68 


0.562 


46 


85 


4.25 


1.093 


75 


4.11 


0.649 


50 


4.20 


0.782 


210 


4.18 


0.841 


59 


85 


2.66 


1.418 


76 


2.55 


1.136 


50 


2.98 


1.237 


211 


2.73 


1.264 


67 


85 


3.18 


1.004 


75 


3.39 


0.999 


50 


3.08 


1.175 


210 


3.21 


1.059 



Table 2. Process Auditing MOSE Component, 
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The Reward System and Safety Culture MOSE component is presented in Table 



3. Ten questions comprise this component, with two that are negatively worded, 
questions 41 and 54. These two questions appear to be a problem, but reverse coding 



shows that they are answered positively (mean range = 3.49 to 4.02, and 3.02 to 3.45 



respectively). The other eight questions are all answered positively (mean range = 3.10 to 
4.32). 



Question 


Fleet Support 
Squadron 


Patrol 

Squadron 


Combat Support 
Squadron 


Combined 

Squadrons 




Total 


Avg 


StdDev 


Total 


Avg 


StdDev 


Total 


Avg 


StdDev 


Total 


Avg 


StdDev 


6 


85 


3.51 


1.158 


76 


3.68 


0.996 


50 


3.82 


1.155 


211 


3.67 


1.103 


7 


85 


3.93 


1.281 


75 


3.99 


0.814 


50 


3.84 


1.167 


210 


3.92 


1.087 


18 


86 


4.24 


0.916 


76 


4.17 


0.870 


50 


3.98 


1.078 


212 


4.13 


0.955 


19 


85 


3.35 


1.874 


76 


4.01 


0.887 


50 


4.08 


0.922 


211 


3.82 


1.228 


36 


84 


3.37 


1.320 


74 


3.20 


1.007 


49 


3.10 


1.141 


207 


3.22 


1.156 


41 


86 


2.51 


1.406 


76 


2.24 


1.118 


50 


1.98 


1.040 


212 


2.24 


1.188 


53 


86 


3.64 


1.198 


76 


3.47 


0.945 


50 


3.56 


1.091 


212 


3.56 


1.078 


54 


85 


2.66 


1.418 


76 


2.55 


1.136 


50 


2.98 


1.237 


211 


2.73 


1.264 


58 


85 


4.14 


1.075 


76 


4.26 


0.719 


49 


4.04 


1.172 


210 


4.15 


0.989 


60 


85 


4.31 


0.715 


76 


4.32 


0.697 


50 


4.32 


0.913 


211 


4.31 


0.775 



Table 3. Reward System and Safety Culture MOSE Component. 

The third MOSE component is Quality. This component has twelve questions, 

with one question, number twenty (mean range = 4.00 to 4.05), requiring reverse coding. 



All questions in this component, except one, are answered positively (mean range = 3.45 



to 4.37). Question 37 (overall mean = 2.82) is answered in a negative manner. This 



question relates to staffing; Is it sufficient from shift to shift. The Quality component 
results are presented in Table 4. 



Question 


Fleet Support 
Squadron 


Patrol 

Squadron 


Combat Support 
Squadron 


Combined 

Squadrons 




Total 


Avg 


StdDev 


Total 


Avg 


StdDev 


Total 


Avg 


StdDev 


Total 


Avg 


StdDev 


9 


86 


4.23 


0.577 


76 


4.20 


0.654 


50 


4.40 


0.571 


212 


4.28 


0.601 


10 


84 


3.68 


1.160 


75 


3.61 


0.769 


50 


3.70 


0.953 


209 


3.66 


0.961 


17 


86 


4.29 


0.632 


75 


4.20 


0.838 


50 


4.28 


0.858 


211 


4.26 


0.776 


20 


86 


1.97 


1.281 


76 


1.95 


1.044 


50 


2.00 


1.161 


212 


1.97 


1.162 


37 


84 


2.63 


1.344 


75 


3.07 


0.991 


50 


2.76 


1.205 


209 


2.82 


1.180 


42 


86 


3.45 


1.757 


74 


3.80 


0.979 


50 


3.80 


1.143 


210 


3.68 


1.293 


44 


86 


4.24 


0.987 


75 


4.28 


0.745 


50 


4.26 


1.006 


211 


4.26 


0.913 


45 


84 


4.32 


1.040 


76 


4.29 


0.745 


50 


4.28 


0.927 


210 


4.30 


0.904 


47 


85 


4.37 


0.830 


76 


4.42 


0.735 


50 


4.66 


0.557 


211 


4.48 


0.707 


48 


84 


3.92 


1.354 


76 


4.26 


0.755 


50 


4.32 


0.768 


210 


4.17 


0.959 


49 


83 


3.82 


1.467 


76 


3.93 


0.971 


50 


3.92 


1.047 


209 


3.89 


1.162 


50 


84 


3.61 


1.567 


76 


4.05 


0.893 


50 


4.16 


1.095 


210 


3.94 


1.185 



Table 4. Quality MOSE Component. 

Risk Management is the fourth MOSE component and it has twleve questions. 



with three questions, numbers 12 (mean range = 2.84 to 3.03), 22 (mean range = 3.55 to 



3.88), and 40 (mean range = 2.93 to 3.12), requiring reverse coding. All but one of the 



questions were answered positively (mean range = 2.98 to 4.33). Question 51 (overall 



mean = 2.78) has negative responses. This question concerns equal workloads and equal 



stress / fatigue between shifts. This negative perception may be of some concern. The 
Risk Management component results are presented in Table 5. 



Question 


Fleet Support 
Squadron 


Patrol 

Squadron 


Combat Support 
Squadron 


Combined 

Squadrons 




Total 


Avg 


StdDev 


Total 


Avg 


StdDev 


Total 


Avg 


StdDev 


Total 


Avg 


StdDev 


11 


84 


2.98 


1.180 


74 


3.00 


0.828 


50 


3.24 


0.847 


208 


3.07 


0.952 


12 


85 


2.97 


1.344 


75 


3.16 


1.027 


49 


2.98 


1.145 


209 


3.03 


1.172 


21 


85 


3.93 


1.019 


76 


3.83 


0.790 


50 


3.98 


0.915 


211 


3.91 


0.908 


22 


86 


2.45 


1.757 


76 


2.12 


0.966 


50 


2.14 


1.125 


212 


2.24 


1.283 


, 23 


85 


3.75 


1.188 


76 


3.67 


0.999 


49 


3.57 


1.155 


210 


3.67 


1.114 


24 


86 


4.33 


0.599 


76 


4.21 


0.754 


50 


4.18 


0.873 


212 


4.24 


0.742 


40 


86 


3.07 


1.619 


74 


2.88 


0.859 


50 


2.88 


1.223 


210 


2.94 


1.234 


51 


84 


2.54 


1.481 


73 


3.18 


0.887 


50 


2.64 


1.156 


207 


2.78 


1.175 


55 


85 


3.46 


1.346 


75 


3.73 


1.004 


50 


3.46 


1.092 


210 


3.55 


1.147 


61 


85 


3.99 


1.107 


76 


4.15 


0.706 


50 


4.12 


0.961 


211 


4.08 


0.925 


62 


84 


4.12 


0.925 


76 


4.20 


0.783 


50 


4.28 


0.784 


210 


4.20 


0.831 


63 


84 


3.76 


1.437 


76 


3.95 


1.018 


50 


4.14 


0.948 


210 


3.95 


1.134 



Table 5. Risk Management MOSE Component. 

Command and Control is the fifth component of the MOSE model. It comprises 



the largest number (14) of questions in the survey, with question 39 worded negatively. 
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Reverse coding question 39 (mean range =2.54 to 2.92) shows that it is responded to 
negatively in the survey. Question 39 deals with passdown between shifts within the 
command. All other questions in this component were answered positively (mean range 
= 2.96 to 4.68). Table 6 tabulates the results for the maintenance survey for the 



Command and Control MOSE component. 



Question 


Fleet Support 
Squadron 


Patrol 

Squadron 


Combat Support 
Squadron 


Combined 

Squadrons 




Total 


Avg 


StdDev 


Total 


Avg 


StdDev 


Total 


Avg 


StdDev 


Total 


Avg 


StdDev 


13 


86 


4.41 


0.621 


76 


4.28 


0.759 


50 


4.32 


0.713 


212 


4.33 


0.698 


14 


85 


3.92 


0.910 


75 


3.93 


0.875 


50 


4.16 


0.817 


210 


4.00 


0.867 


25 


86 


4.00 


0.918 


76 


4.05 


0.798 


50 


4.08 


0.900 


212 


4.04 


0.872 


26 


86 


3.90 


1.036 


76 


4.13 


0.660 


50 


4.04 


0.968 


212 


4.02 


0.888 


27 


86 


3.91 


0.932 


75 


4.08 


0.749 


50 


3.92 


0.966 


211 


3.97 


0.882 


28 


86 


3.94 


0.950 


76 


3.95 


0.798 


50 


3.90 


0.886 


212 


3.93 


0.878 


29 


86 


3.71 


1.173 


75 


3.27 


1.212 


50 


3.76 


1.153 


211 


3.58 


1.179 


30 


86 


3.81 


1.259 


76 


4.08 


0.707 


50 


4.06 


0.998 


212 


3.98 


0.988 


38 


85 


3.82 


1.242 


76 


3.72 


1.015 


50 


3.74 


1.026 


211 


3.76 


1.094 


39 


85 


3.35 


1.541 


75 


3.08 


0.941 


50 


3.46 


1.129 


210 


3.30 


1.204 


56 


85 


4.45 


0.703 


76 


4.47 


0.621 


50 


4.68 


0.551 


211 


4.53 


0.625 


64 


85 


4.01 


0.964 


76 


4.00 


0.766 


50 


4.16 


0.842 


211 


4.06 


0.857 


65 


84 


3.81 


1.264 


76 


3.96 


0.807 


50 


4.14 


0.904 


210 


3.97 


0.992 


66 


83 


2.96 


1.206 


75 


3.07 


0.920 


50 


3.14 


0.990 


208 


3.06 


1.039 



Table 6. Command and Control MOSE Component. 

The final component of the MOSE is Communication / Relationships. This 



component has eleven questions, with question eight (mean range = 3.23 to 3.70) worded 



negatively. Questions 34 (overall mean =2.99) and 43 (overall mean = 2.64) show some 



concern, resulting in response values less than three. The Communication / Relationship 



component is tabled in Table 7. 
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Question 


Fleet Support 
Squadron 


Patrol 

Squadron 


Combat Support 
Squadron 


Combined 

Squadrons 




Total 


Avg 


StdDev 


Total 


Avg 


StdDev 


Total 


Avg 


StdDev 


Total 


Avg 


StdDev 


8 


84 


2.77 


1.575 


75 


2.39 


1.102 


50 


2.30 


1.165 


209 


2.49 


1.281 


15 


86 


3.63 


1.531 


76 


3.33 


1.038 


50 


3.28 


1.161 


212 


3.41 


1.243 


16 


85 


3.86 


0.956 


74 


3.53 


0.954 


50 


3.68 


0.794 


209 


3.69 


0.901 


31 


86 


3.73 


1.257 


76 


3.40 


1.034 


50 


3.52 


1.216 


212 


3.55 


1.169 


32 


85 


4.01 


1.083 


76 


4.34 


0.684 


50 


3.88 


1.154 


211 


4.08 


0.974 


33 


86 


3.92 


0.970 


75 


3.96 


0.892 


50 


3.94 


0.998 


211 


3.94 


0.953 


34 


86 


2.90 


0.918 


73 


3.16 


1.028 


50 


2.90 


1.055 


209 


2.99 


1.000 


35 


85 


3.65 


1.612 


75 


3.43 


1.029 


50 


3.32 


1.115 


210 


3.46 


1.252 


43 


84 


2.80 


1.440 


75 


2.48 


1.178 


50 


2.64 


1.174 


209 


2.64 


1.264 


52 


86 


3.79 


1.203 


75 


3.99 


0.762 


50 


3.92 


0.900 


211 


3.90 


0.955 


57 


85 


3.94 


1.151 


76 


4.03 


0.730 


50 


4.10 


1.035 


211 


4.02 


0.972 



Table 7. Communication / Relationship MOSE Component. 

Each question within the MOSE is answered positively for all participating 



squadrons. The average for all questions in each MOSE component was computed and is 



tabulated in Table 8. 



MOSE Component 


Fleet Support 
Squadron 


Patrol 

Squadron 


Combat Support 
Squadron 


Combined 

Squadrons 


Process Auditing 


3.97 


3.94 


3.94 


3.95 


Reward System 


3.73 


3.83 


3.78 


3.78 


Quality 


3.88 


4.01 


4.05 


3.98 


Risk Management 


3.52 


3.67 


3.63 


3.61 


Command & Control 


3.81 


3.85 


3.90 


3.85 


Comm/Functional Relationships 


3.59 


3.57 


3.41 


3.56 



Table 8. MOSE Component Summary Results. 



C. ANALYSIS PROBLEM 

The dataset used in this portion of the analysis uses only the Maintenance surveys. 
Of the total 268 surveys collected, 212 were from Maintainers. These 212 responses are 
used to form a data matrix, X, to be studied and analyzed using the S-Plus statistical 
package. If X is n by k, then n = 212, the total number of survey respondents and k = 67, 
the number of survey questions presented. 

The problem is to reduce the existing k MCAS items to a more reasonable number 



for a full-scale survey. An acceptable number of questions for a survey is in the range of 
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25 to 40 questions (Fink, 1985). Clustering methods such as agglomerative nesting and 
divisive analysis are sensible tools for such problems. 

These methods belong to the family of hierarchical clustering methods that 
produce a tree-like hierarchy. This structure is most conveniently visualized through a 
dendrogram such as the one depicted in Figure 6. The vertical scale is a distance measure 
that marks the levels at which two clusters are joined to make one. The horizontal is a 
nominal scale on which the item designators are permuted so that the tree arcs do not 
cross one another. From the bottom up we can view how the k items are joined to form 
successive clusters. The use of such information can help provide a dispassionate guide 
for grouping the 67 items into a more reasonably sized set. 

D. AGGLOMERATIVE NESTING (AGNES) 

Two S-PLUS algorithms are used, AGNES and DIANA. Agglomerative nesting 
(AGNES) is discussed first as it is the easier to describe. It is a “bottom up” approach so, 
initially, there are (k=67) clusters; each single item is viewed as a cluster. First, one 
selects a distance measure. Suppose euclidian distance is chosen and is initially 
computed as the distance between each pair of questions. One finds the smallest value of 
this set and forms a cluster of the two items associated with it. Now, there are k-1 
clusters instead of k, and the distance of all items from the new cluster of two must be 
computed and used to replace the 2 * (k-2) distances of the two selected items from all 
others that have been supplanted. They are needed no more but there is a choice in 
defining the new distances. It was chosen to average the distances between all pairs of 
items, one from each cluster, to obtain a distance figure separating two clusters. This 
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done, again the smallest distance is chosen for forming the next new cluster, etc. The 
process continues until there is but one cluster consisting of all k items. The user can 
select the clustering solution that suits him/her. Scanning the vertical scale is often useful 
for this purpose. 

The agglomerative nesting function called “AGNES” in S-PLUS package is used 
as a first method to cluster the k questions or variables of the Maintenance safety survey 
into fewer questions. The S-PLUS function “AGNES” is called using the command 
agnes(daisy (t(X) ) ,diss=T) . The S-PLUS function “daisy” is used to calculate 
the dissimilarity matrix for X. The “diss=T” statement means that a dissimilarity matrix 
is to be used in the computations to serve as the distance function. This choice 
corrresponds to the one described. 

Results from AGNES are then plotted using the pltree.agnes S-PLUS function. 
The pltree.agnes function produces a clustering tree of agglomerative hierarchical 
clustering when it is passed an AGNES object. It creates a plot of a clustering tree. The 
leaves of the tree are the original objects or in this case, the k questions. Two branches of 
the tree come together at the distance between the two clusters being merged. Figure 6 
shows the clustering tree resulting from the pltree.agnes command. 
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Figure 6. AGNES Clustering Tree. 

The AGNES function returns an object containing the following items: order, 
height, agglomerative coefficient, and a matrix called merge. Order is a vector giving a 
permutation of the original objects to allow for plotting, in the sense that the branches of 
a clustering tree will not cross. Height is a vector with the distances between merging 
clusters at the successive stages. The agglomerative coefficient (ac) measures the amount 
of clustering structure found in the dataset. More explicitly to each item i, denoted by 
m(i), the dissimilarity to the first cluster it is merged with, is divided by the dissimilarity 
of the merger in the final step of the algorithm. The ac value is the average of all the { 1- 
m(i)}. The resulting ac value produced by AGNES is 0.5445873. The AGNES object 
has a resulting merge matrix, describing the merging of clusters at steps within the 
clustering process. It provides the numerical support for the dendrogram. 
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Merge is a (k-1) by 2 matrix, where each row i of merge describes the split at step 
k-i of the clustering. The merge item returned using AGNES is important. This matrix is 
a road map for reading the clustering tree plot resulting from the pltree.agnes function. 
This matrix and the tree plot are used as an aid in clustering the questionnaire items. 
First, pairs are clustered utilizing the interpretations of the merge matrix. Next, triple and 



quad clusters are identified. Easily identified pairings, triples and quads are shown in the 



Table 9. Also shown in Table 9 are the singles, eight questions that do not merge with 



previously constructed clusters until high in the tree. 



Singles 


Pairings 


Triples 


Quads 


7 


43 


2,14 


26,27 


44,45 


10,16 


37,51 


1,9,13 


11,66,67 


44,45,46,48 


19 


52 


3,4 


29,31 


46,48 


18,24 


39,40 


6,29,31 


12,39,40 


8,20,22,41 


32 


53 


5,47 


30,33 


49,50 


22,41 


57,58 


10,16,38 


56,60,62 


19,35,42,55 


35 


63 


9,13 


34,36 


56,62 


42,55 


61,65 


61,64,65 


25,26,27,28 




66,67 


. 



Table 9. AGNES Clustering. 

From these identified pairings, additional items were then added by using the 
corresponding merge information until all questions were covered. Table 10 shows the 
remaining questions and how they were added to the initial clusters. It shows the 



questions, identified by the AGNES algorithm, that are near in distance. 



(2,14) 


(1,5,9,13,17,47,56,60,62) 


(11,34,36,66,67) 


(46,48) 


(37,51) 


(3,4) 


(6,10,15,16,23,29,31,38) 


(8,20,22,41,54) 


(49,50) 


(39,40) 


(30,33) 


(44,45,57,58,59,61 ,64,65) 


(18,21,24,25,26,27,28) 


(42,55) 





Table 10. Additional AGNES Clustering. 



E. DIVISIVE ANALYSIS (DIANA) 



The AGNES algorithm is a bottom up approach to hierarchical clustering. The 



DIANA algorithm reverses this process. It is top down and the resulting tree need not be 
the same. The distance between clusters is used for explanatory purposes. 
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The process begins with a single cluster of all items and then considers all 
possible partitions of the items into two clusters. (It is convenient to think of it in this 
way, although efficient algorithms exist which do not actually do this.) The pair of 
clusters selected is the one with the greatest separation. Then the process is repeated using 
each of the clusters so formed, etc. Ultimately, the cluster will consist entirely of single 
items. The dendrogram for this method appears in Figure 7. Comparing Figure 6 and 
Figure 7 it is seen that the trees are quite similar. This is a comforting check. 

The divisive analysis function called “DIANA” in the SPLUS package is used as 
an approach to “cluster” the k questions into a tree. The DIANA function in S-PLUS is 
called using diana (daisy (t (X) ) , diss=T) , where X represents the data matrix 
used in AGNES. Again, the S-PLUS function “daisy” is used to calculate the 
dissimilarity matrix for X. The “diss=T” statement means that the same dissimilarity 
matrix is to be used in the distance computations. 

Results from DIANA are then plotted using the pltree. diana function to look at the 
clustering results. The S-PLUS function pltree. diana creates a plot of a clustering tree or 
dendrogram given a DIANA object. The leaves of the tree lead to the k original objects or 
questions. A branch splits up at the diameter of the cluster being split. Figure 7 shows the 
resulting tree of “clusters” from the DIANA analysis. 
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Figure 7. DIANA Clustering Tree. 

The DIANA function returns the following items: “order”, “height”, the “divisive 
coefficient”, and “merge”. Order is a vector giving a permutation of the original objects to 
allow for plotting, in the sense that the branches of a clustering tree will not cross. Height 
is a vector with the diameters of the clusters prior to splitting. The divisive coefficient 
measures the amount of clustering found. More explicitly, for each item i, denoted by 
d(i), the diameter of the last cluster to which it belonged ( prior to being split as a single 
object ), divided by the diameter of the whole set. The dc measure is the average of all 
{ l-d(i)}. The divisive coefficient resulting from DIANA is found to be 0.6559955. Like 
the AGNES object, the DIANA object has a resulting merge matrix, describing the 
merging of clusters at steps within the clustering process. 

Again, the merge matrix and the tree plot are used as an aid in clustering the 
questionnaire items. First, pairs clustered are identified and analyzed to see if they make 
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sense. Next, triple and quad clusters are identified. Easily identified pairing are shown in 



the Table 1 1. 



Singles 


Pairings 


Triples 


Quads 


19 


2,14 


12,40 


30,33 


56,60 


1,9,13 


12,40,43 


11,34,37,51 


23 


3,4 


18,24 


37,51 


57,58 


10,16,38 


20,22,41 


19,42,55,53 


35 


5,47 


22,41 


42,55 


61,65 


15,29,31 


30,32,33 


25,26,27,28 


39 


9,13 


25, 26 1 


44,45 


62,64 


17,18,24 




44,45,46,48 


43 


10,16 


27,28 


46,48 


66,67 






61 ,62,64,65 


53 


11,34 


29,31 


49,50 











Table 11 DIANA Clustering. 



From these identified pairings, additional items are then added using the 



corresponding merge information until all questions are covered. Table 12 shows the 



remaining questions and where they are added to the initial pairings. It shows the 



questions, identified by the DIANA algorithm, that are near in distance. 



(1,5,9,13,47,56,60) 


(57,58,59,61,62,63,64,65) 


(8,20,22,41,54) 


(6,10,16,23,38) 


(3,4) 


(11,12,34,37,40,43,57) 


(15,29,31,35,36,39,66,67) 


(30,32,33,52) 


(2,7,14) 


(49,50) 


(44,45,46,48) 


(21,25,26,27,28) 


(19,42,53,55) 


(17,18,24) 





Table 12. Additional Clustering with DIANA. 



Comparing the two S-PLUS methods of AGNES and DIANA shows some 
interesting aspects. First, by comparing the two clustering trees, the shape of each tree is 
very similar. The initial pairings found in the two methods are alike also. Of the 23 
initial pairings from DIANA and the 21 pairings from AGNES, 17 are common to both. 
Moving up the clustering tree one level, more similarities are seen. Furthermore, by 
looking at higher levels of the two clustering trees, clusters of three or four questions 
form. Table 13 shows the common pairings, triples and quads found between the two 
methods. 
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Pairings 


Triples 


Quads 


2,14 


9,13 


22,41 


37,51 


46,48 


1,9,13 


25,26,27,28 


3,4 


10,16 


29,31 


42,55 


49,50 


10,16,38 


8,20,22,41 


5,47 


18,24 


30,33 


44,45 


57,58 


61 ,64,65 


44,45,46,48 


61,65 


66,67 





Table 13. Common Clusters Between DIANA and AGNES. 



F. PRINCIPAL COMPONENTS / FACTOR ANALYSIS 

The Principal Component / Factor Analysis material will help complete the 
questionnaire size reduction process. Principal Component Analysis was conducted on 
the dataset using the S-PLUS software package. The S-PLUS command of 
princomp(X,scores=T,cor=T) was used in the analysis. X is the 212 by 67 matrix of 
survey responses and “cor=T” means that the principle component analysis is to be based 
on the correlation matrix. The resulting principle component output was plotted using a 
screeplot, a special S-PLUS barplot function for the class princomp. The S-PLUS 
function screeplot(prin2.out) was used. It shows the contribution of each factor to the 
total variance. Figure 8 shows the resulting screeplot. 
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Figure 8. Principal Component Screeplot. 

Above each bar in the plot are the cumulative fractions of the variance. The 

variance is sharply the highest on the first component and spread rather evenly over the 
remaining components. This shows evidence that the survey data is dominated by one 
dimension. Much of the variability is captured in the first component. This and Factor 
Analysis is used to assist in answering research questions three and five. 

Factor analysis on the data is performed using S-PLUS. The S-PLUS command 
of factanal(X,factors=6,method=”principal”, rotation=”varimax”) was used. Again, X is 
the 212 by 67 matrix of survey responses, six factors were fit, using varimax rotation as 
the method of aligning the factors in a six dimensional space. The “factors=6” input 
specifies the number of factors in the solution. An object of class factanal results. 

The factanal class results from fitting factor analysis models. This class has 
certain properties that are relevant to analyze. An important property is “loadings.” 
Loadings is an orthogonal matrix that gives the loadings for each factor, each column 
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being linear combinations of the columns of X for the corresponding factor. Another 
property of the factanal class is uniqueness. The variance of each item is decomposed 
into the part that is common to the factor solutions and the part that is unique to that item. 
The loadings and uniquenesses for each question in the corresponding components of the 
MOSE are presented in Tables 14 through 19. 



Question 


Factor 1 


Factor 2 


Factor 3 


Factor 4 


Factor 5 


Factor 6 


Uniqueness 


1 


0.462 


0.306 




0.149 


0.176 




0.6381 


2 


0.337 


0.278 


0.189 


0.219 


0.263 




0.6553 


3 


0.525 


0.281 


- 0.133 


0.102 


0.268 


- 0.147 


0.5263 


4 


0.452 


0.214 




0.131 


0.230 


- 0.140 


0.6555 


5 


0.478 


0.107 




0.167 




0.195 


0.6862 


46 


0.612 


0.117 


0.146 








0.5897 


59 


0.234 


0.596 


0.243 


0.133 


0.148 




0.4885 


67 


0.143 


0.339 


0.436 




0.152 




0.6506 



Table 14. Factor Loadings for Process Auditing Component. 



Question 


Factor 1 


Factor 2 


Factor 3 


Factor 4 


Factor 5 


Factor 6 


Uniqueness 


6 


0.273 


0.216 


0.378 


0.207 


0.338 




0.5693 


7 


0.401 


0.287 


0.180 


0.267 


0.322 




0.5456 


18 


0.459 


0.313 




0.297 


0.384 




0.4447 


19 


0.366 




0.128 


0.139 






0.8288 


36 




0.314 


0.117 


0.329 


0.441 


- 0.160 


0.5576 


41 


- 0.402 


- 0.213 


0.116 


- 0.253 




- 0.455 


0.5007 


53 


0.239 


0.131 




0.202 


0.277 


- 0.105 


0.7925 


54 


- 0.179 


- 0.188 




- 0.372 




- 0.165 


0.7532 


58 


0.249 


0.593 


0.152 


0.310 


0.149 




0.4392 


60 


0.282 


0.625 


0.112 


0.204 


0.224 


0.133 


0.4073 



Table 15. Factor Loadings for Reward and Safety Culture. 
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Question 


Factor 1 


Factor 2 


Factor 3 


Factor 4 


Factor 5 


Factor 6 


Uniqueness 


9 


0.507 


0.207 


0.200 




0.227 




0.5898 


10 


0.440 


0.141 


0.373 




0.284 


- 0.100 


0.5573 


17 


0.617 


0.221 




0.133 


0.250 


0.101 


0.4796 


20 


- 0.222 


- 0.327 


0.116 


- 0.258 




- 0.464 


0.5402 


37 




0.118 


0.537 




0.184 


0.121 


0.6469 


42 


0.276 


0.147 


0.412 


0.121 




0.245 


0.6564 


44 


0.520 


0.239 


0.124 


0.189 




0.181 


0.5830 


45 


0.469 


0.159 


0.154 






0.152 


0.6991 


47 


0.657 


0.188 










0.5148 


48 


0.567 


0.115 


0.461 


0.143 






0.4258 


49 


0.425 


0.215 


0.238 


0.250 


0.214 




0.6017 


50 


0.401 


0.333 


0.410 


0.205 




0.152 


0.4908 



Table 16. Factor Loadings for Quality. 



Question 


Factor 1 


Factor 2 


Factor 3 


Factor 4 


Factor 5 


Factor 6 


Uniqueness 


11 




0.129 


0.343 




0.114 


- 0.275 


0.7637 


12 








0.166 




- 0.365 


0.8236 


21 


0.572 


0.158 




0.226 


0.256 




0.5284 


22 


- 0.243 


- 0.146 




- 0.293 




- 0.473 


0.6092 


23 


0.162 


0.252 


0.149 




0.363 


0.144 


0.7275 


24 


0.487 


0.33 




0.415 


0.194 




0.4371 


40 






- 0.128 






- 0.445 


0.7717 


51 






0.496 


0.178 




0.101 


0.6977 


55 


0.19 


0.33 


0.414 


- 0.112 


0.193 


0.312 


0.5364 


61 


0.239 


0.638 


0.321 


0.157 


0.176 




0.3709 


62 


0.365 


0.591 


0.305 


0.135 


0.192 


0.12 


0.3550 


63 


0.372 


0.418 


0.225 


0.246 


0.178 


0.167 


0.5159 



Table 17. Factor Loadings for Risk Management. 



Question 


Factor 1 


Factor 2 


Factor 3 


Factor 4 


Factor 5 


Factor 6 


Uniqueness 


13 


0.483 


0.240 




0.158 


0.272 


0.183 


0.5747 


14 


0.415 


0.280 


0.287 


0.101 


0.256 


0.104 


0.5800 


25 


0.330 


0.301 


0.165 


0.553 


0.131 


0.109 


0.4384 


26 


0.328 


0.338 


0.258 


0.609 






0.3318 


27 


0.417 


0.155 


0.218 


0.642 


0.166 




0.3063 


28 


0.376 


0.327 


0.206 


0.563 


0.147 




0.3690 


29 


0.338 


0.198 


0.213 


0.176 


0.523 




0.4972 


30 


0.388 


0.248 


0.316 


0.312 


0.246 


0.165 


0.5029 


38 


0.378 


0.131 


0.335 




0.454 


0.131 


0.5032 


39 




0.107 


- 0.108 






- 0.372 


0.8253 


56 


0.314 


0.485 


0.229 


0.183 


0.108 


0.195 


0.5305 


64 


0.238 


0.609 


0.211 


0.231 


0.193 




0.4305 


65 


0.210 


0.587 


0.387 


0.141 


0.103 




0.4293 


66 


0.158 


0.196 


0.579 




0.122 




0.5808 



Table 18. Factor Loadings for Command and Control. 
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Question 


Factor 1 


Factor 2 


Factor 3 


Factor 4 


Factor 5 


Factor 6 


Uniqueness 


8 


- 0.160 


- 0.166 


- 0.107 


- 0.268 


- 0.101 


- 0.295 


0.7663 


15 


0.136 


0.264 


0.337 


0.159 


0.521 


0.120 


0.4876 


16 


0.449 


0.263 


0.199 




0.441 




0.4908 


31 


0.206 


0.226 


0.248 


0.279 


0.621 




0.3735 


32 


0.339 


0.142 


0.252 


0.307 


0.122 




0.6877 


33 


0.336 


0.417 


0.156 


0.372 


0.191 




0.5109 


34 




0.122 


0.232 


0.320 


0.317 


- 0.145 


0.7043 


35 


0.300 








0.348 




0.7644 
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0.166 




0.9669 


52 


0.441 


0.181 


0.308 


0.208 






0.6322 


57 


0.185 


0.585 


0.122 


0.195 


0.220 


0.228 


0.4701 



Table 19. Factor Loadings for Commuinication / Relationships. 



For each component, the questions having the maximum loading over the six 
factors are analyzed. As true in the principle component analysis, there is one factor, 
Factor one, which had the most loadings having the maximum value. This analysis, with 
results shown in Table 20, shows the relationships between the factors resulting from the 
factor analysis and the MOSE components. Of the 67 questions, 25 of them load the 
highest in factor one. This provides quantitative information on these items that are 
represented in the single most important factor dimension. 





Factor 1 


Factor 2 


Factor 3 


Factor 4 


Factor 5 


Factor 6 


Component 1 


1,2,3,4,5,46 


59 


67 








Component 2 


7,18,19 


58,60 


6 


54 


36,53 


41 


Component 3 


9,1 0,1 7,44,45,47,48,49 




37,42,50 






20 


Component 4 


21,24 


61,62,63 


11,51,55 




23 


12,22,40 


Component 5 


13,14,30 


56,64,65 


39,66 


25,26,27,28 


29,38 




Component 6 


16,32,52 


33,57 


15 


34 


15,16,31,34,35,43 


8 



Table 20. Questions and Factors Loadings Broken Out by MOSE Component. 



Table 21 shows the distribution of the questions over the Factors and MOSE 
components resulting from the factor analysis. Interesting to note is the number of 
questions that belong to factor 1. Twenty Five of the 67 questions in the MCAS fall into 
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the first factor. The remaining factors have loadings with various signs and provide 
contrast type information about the items that dominate them. 





Factor 1 


Factor 2 


Factor 3 


Factor 4 


Factor 5 


Factor 6 


Component Totals 


Component 1 


6 


1 


1 


0 


0 


0 


8 


Component 2 


3 


2 


1 


1 


2 


1 


10 


Component 3 


8 


0 


3 


0 


0 


1 


12 


Component 4 


2 


3 


3 


0 


1 


3 


12 


Component 5 


3 


3 


1 


4 


2 


1 


14 


Component 6 


3 


2 


0 


1 


4 


1 


11 


Factor Totals 


25 


11 


9 


6 


9 


7 


67 



Table 21. Distribution of Questions From Factor Analysis. 



Also interesting to study is the cumulative distribution function of the 
uniquenesses for each question resulting from the factor analysis. This is important to 
study because the questions with low uniqueness in the large, have their content 
represented by other questions. The questions with high uniqueness have individual 
dominating status. From this, one may be able to identify questions that can be 
eliminated because they may be non-contibutors. Also, we can identify questions of high 
uniqueness, which either should be retained as singletons, or identified as dominating the 
group to which they belong. Figure 9 is the resulting S-PLUS CDFDISC function plot of 
the sorted uniquenesses and associated probabilities (Appendix E shows the S-PLUS 
CDFDISC function code). 
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Uniqueness 

Figure 9. Question Uniqueness Empirical CDF. 

Looking at the plot above gives a good view of the uniqueness for the questions. 

Note the flat spot appearing in Figure 9 at the 0.4 uniqueness level on the x-axis. The 
questions relating to these uniqueness values are possible candidates for elimination since 
they all have relatively low uniqueness. Below is Table 22 showing the seven questions 
whose uniqueness fall near or below the 0.4 value. 



Question 


Uniqueness 


26 


0.3318 


27 


0.3063 


28 


0.3690 


31 


0.3735 


60 


0.4073 


61 


0.3709 


62 


0.355 



Table 22. Questions with Low Uniquenesses. 

There is also a flat area Figure 9 at the 0.75 uniqueness level on the axis 



identifying questions with high uniqueness relative to the entire set. All uniquenesses to 



the right of 0.75 relate to those having a relatively high uniqueness. There are ten such 



questions. Table 23 identifies the ten questions showing the uniqueness values for each. 
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Question Uniqueness 


8 


0.7663 


11 


0.7637 


12 


0.8236 


19 


0.8288 


35 


0.7644 


39 


0.8253 


40 


0.7717 


43 


0.9669 


53 


0.7925 


54 


0.7532 



Table 23. Questions with High Uniqueness. 

All of this information is useful to aid the researcher in making informative 



decisions about the items in the data. Each of the above areas of analysis are used in 



conjunction with one another to obtain the goal of a reduced set of questions to be used in 



an improved and modified MCAS. 



G. QUESTION ANALYSIS 

The S-Plus functions AGNES, DIANA, PRINCOMP and FACT ANAL assist the 
researcher in making decisions or objective choices about the items. These will be 
viewed for reasonableness based on subjective knowledge. The stability of these choices 
and the results from the analysis are also influenced by the relatively small sample size 
involved in this study. With this in mind, the questions are studied using the computer- 
aided output from the previous sections. The prototype MCAS had questions that 
required reverse coding. It is recommended these questions be rephrasedto read in a 
positive aspect. (Appendix H lists these questions and the recommended modificatiQns.) 

Decision rules need to be identified as to how the downsizing of the MCAS is 
approached. First, the seven questions identified with low uniqueness are removed. 
Likewise, questions identified, as having high uniqueness, will be retained. Secondly, all 
pairs that are the same in both AGNES and DIANA are studied for combination or to see 
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if the questions can be reworded to make one out of the two. This done, the next step is 
to take all triples and then quadruples identified by the clustering algorithms and look for 
sensible combinations or removal of questions. At the completion of this, the remaining 
questions that have not been grouped or clustered will be studied for further sensible 
reduction. 

Questions having low uniqueness are eliminated. There are seven such questions 
that have been identified in Table 22. The elimination of these seven questions results in 
a remaining pool of 60 questions. Furthermore, we retain the ten questions identified in 
Table 23 that have high uniqueness. This leaves 50 questions for continued analysis. 
Next, the cluster analysis results from AGNES and DIANA are used for combining 
questions that are answered in similar ways or show redundancy. 

Questions paired up that are common to both the AGNES and DIANA methods 
are analyzed to see if the pairings can be combined easily. Here, questions are paired up 
due to their relative closeness resulting from the computer-aided output. Questions are 
studied and if sensible, can be joined with the conjunction “and” or reworded to capture 
the intent of the original questions. For example, the first pairing from Table 13 is the 
pair (9,13). By rewording these questions slightly, a new question results that replaces 
the two questions with one. 

This process is done for all identified clusters in Table 13 as well as possible 
clusters from Tables 9 and 11. Table 24 lists these groupings. Appendix F gives the 
suggested rewording of these newly formed questions to be included in the suggested 
MCAS. 
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Nineteen questions are suggested as candidates for combining, rewording or 
grouping that reduce the MCAS question total by an additional 24 questions. A sensible 
pairing is made from two questions having high uniqueness. DIANA identifies pair 
(19,53) as a cluster and AGNES shows agreement in that the dendrogram has the pairing 



in close proximity to one another. This pair belongs to the same MOSE component and 



the two questions have the same intent, thus the two are combined into one. Question 1 1 



also had high uniqueness but it was sensible to cluster it as noted in Table 24. 



1,9,13 


11,36,59,67 


20,22 


49,50 


2,14 


10,66 


30,33 


52,56 1 


3,4 


16,38 


42,55 


57,58 


5,47 


18,21,24 


37,51 


64,65 


6,23 


19,53 


45,46,48 





Table 24. Suggested Question Groupings. 



Nine singletons result in this process. These questions neither clustered with 
others nor seemed reasonable to combine with any other question in the MCAS. These 
singletons are presented below in Table 25. 



7 


25 


41 


15 


29 


44 


17 


34 


63 



Table 25. Original Questions Remaining. 

By combing the results in the first step, deleting the questions with relatively low 
uniqueness and then combining questions using the cluster analysis results, the MCAS 
reduces to 35 questions; seven from the high uniqueness, 19 from the clustering or 
rewording, and an additional nine as singletons. There is a judgement call here, and 
given the data and the questions, this seems to be as far as one can go without adding 
additional scrutiny to the study. Table 26 gives the suggested mapping of the 67-question 
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prototype MCAS to a suggested 35-question MCAS. The P-Type column contains the 
ordinal number of the item in the original 67-question MCAS, and the Mapped column 
displays the ordinal number in the suggested 35-question MCAS. 



P-Type 


Mapped 


P-Type 


Mapped 


P-Type 


Mapped 


P-Type 


Mapped 


P-Type 


Mapped 


P-Type 


Mapped 


P-Type 


Mapped 


1 


6 


11 


8 


21 


13 


31 


Deleted 


41 


25 


51 


22 


61 


Deleted 


2 


1 


12 


9 


22 


15 


32 


Deleted 


42 


26 


52 


31 


62 


Deleted 


3 


2 


13 


6 


23 


16 


33 


19 


43 


27 


53 


14 


63 


34 


4 


2 


14 


1 


24 


13 


34 


20 


44 


28 


54 


32 


64 


35 


5 


3 


15 


10 


25 


17 


35 


21 


45 


29 


55 


26 


65 


35 


6 


16 


16 


11 


26 


Deleted 


36 


8 


46 


29 


56 


31 


66 


7 


7 


4 


17 


12 


27 


Deleted 


37 


22 


47 


3 


57 


33 


67 


8 


8 


5 


18 


13 


28 


Deleted 


38 


11 


48 


29 


58 


33 




9 


6 


19 


14 


29 


18 


39 


23 


f ■} 


30 


59 


8 


10 


7 


20 


15 


30 


19 


40 


24 


50 


30 


60 


Deleted 



Table 26, Prototype MCAS Questions Mapped to Suggested MCAS Questions, 



V. CONCLUSION 



A. FINDINGS 

The existing MOSE can be used to model or capture the maintained s perspective, 
with modifications presented in this thesis. The close-ended questions, after tailoring 
them towards the maintainer, conformed to the existing MOSE components quite readily. 
However, an additional component. Communication / Relationships, was added to make 
the MOSE maintenance specific. Results of the prototype MCAS were positive in nature 
for all six of the adapted MOSE components except for a few questions that raised 
concern and these are listed in Table 27. These questions may possibly identify areas that 



need strengthening within the squadrons. 



Question 


Fleet Support 
Squadron 


Patrol 

Squadron 


Combat Support 
Squadron 


Combined 

Squadrons 


59 


2.66 


2.55 


2.98 


2.73 


37 


2.63 


3.07 


2.76 


2.82 


51 


2.54 


3.18 


2.64 


2.78 


39 


2.65 


2.92 


2.54 


2.7 


34 


2.9 


3.16 


2.9 


2.99 


43 


2.8 


2.48 


2.64 


2.64 



Table 27. Questions That Raise Concern. 

There are no discemable differences identified between the squadrons that were 



surveyed. A consenus between all participating squadrons was that of favorable results 
for all MOSE components. Agglomerative nesting and Divisive Analysis together with 
Prinicpal Components and Factor Analysis were tools used to identify clusters of 
questions that were similar. Although the Factor Analysis shows that most of the 
questions load on factor one, there is no evidence against the structure of the maintenance 



adapted MOSE. Redundant questions were eliminated and questions with similar intent 

were combined. Thus, the prototype MCAS was reduced from 67 to 35 questions, 
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shortening the survey to a more practical length, requiring less time to complete. The 
finalized MCAS is listed in Appendix I. 

B. OTHER AREAS OF CONCERN 

There are numerous future studies that can further this thesis. One such task 
would be to alter the questionnaire in order to use it with the Marine Corps, using 
“Marine” specific terms. Also, this study could be further investigated by surveying 
active duty Naval Aviation Squadrons to make comparisons of the results with the 
Reserve Squadron results presented in this study. The Marine Squadron results could be 
compared to Navy Squadrons to see if there is a statistical difference between the two. 
Another task would be to administer the survey to Regular active duty Squadrons and do 
a comparison between Reserve and active duty Squadrons. Also, other analysis methods 
could be used to validate this study and its results. 

C. SUMMARY 

The Class A flight mishap rate has leveled off over the past decade. Human 
Factors studies are increasing as they tyy to identify additional areas where intervention 
could lead to further reductions of the Class A flight mishap rate. This thesis is part of a 
Human Factors related study specifically aimed at Maintenance operations within Naval 
Aviation. The resulting survey is to be used as a tool for commanders to assess the 
effectiveness of Naval Aviation maintenance operations in the management of risk. 
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APPENDIX A. EXISTING MOSE COMPONENTS 



COMPONENT 1: PROCESS AUDITING 

1 . Conducts periodic safety reviews and inspections. 

2. Conducts timely reviews and updates of safety practices and operating standards. 

3. Uses a systematic process to set training goals and to review flight qualifications. 

4. Uses Human Factors Councils and Boards to identify and screen high-risk aviators. 

5. Makes effective use of Flight Surgeon for assessing significant stress reactions, and 
possible unsafe attitudes of individuals as well as routine medical screening. 

COMPONENT 2: REWARD SYSTEM 

1. Recognizes safety achievement through social praise, and formal awards and 
incentives. 

2. Safe behavior is reinforced as a cultural norm (by command emphasis and peer 
pressure). 

3. Takes timely action (s) to appropriately discipline unsafe behavior and attitudes. 

4. Encourages everyone to be safety conscious and to identify safety problems. 

5. Encourages everyone to .report safety discrepancies without fear of negative 
repercussions. 

COMPONENT 3: QUALITY 

1. Sets high quality standards as an organizational goal. 

2. Publishes quality standards and quality control procedures. 

3. Monitors quality and corrects deviations from established quality standards. 

4. Gains reputation for high-quality performance in comparison to reference standards or 
comparable organizations. 

COMPONENT 4: RISK MANAGEMENT 

1 . Has accurate perception of actual operational mission risks. 

2. Uses a systematic method for managing risk. 

3. Risks of mission operations are acknowledged and minimized. 

4. Safety risk decisions are made at the proper level (by most qualified people). 

5. Resources (time, budget, staffing, and equipment) are adequate for performance of 
mission and organizational workload. 

6. Does not compromise safety to get the work done or to accomplish the mission. 
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COMPONENT 5: COMMAND AND CONTROL 



1. Supervisors openly demonstrate a commitment to safety. 

2. Supervisors clearly communicate safety goals and policies. 

3. Supervisors are directly involved in safety management. 

4. Safety is considered as an integral part of mission accomplishment. 

5. The command establishes and uniformly enforces operating standards. 

6. There is frequent interaction between command supervisors and staff. 

7. There is good communication flow up and down the command chain. 

8. Safety training is emphasized throughout the command. 

9. The status and respect for the safety officer's position are high. 

10. Personnel turnover is not high enough to affect a command's ability to maintain 
"corporate" knowledge and the core cultural values of the organization. 

11. Supervisors provide adequate guidance and counseling on safety matters. 

12. Formal rules and procedures are followed, whether or not command supervisors are 
physically present. 

13. There is redundancy, multiple coverage or adequate safety backups for high-risk 
operations. 

14. Personnel in the command conduct continuous proficiency training. 

15. Training incorporates safety guidelines and safeguards. 

16. Leadership has vision and understands how to create a positive climate. 

17. The prevailing values, beliefs, attitudes, and norms (culture) promote safe 
behavior/discourages unsafe behavior. 

18. Leadership sets example for written policy, standards, proper procedures, and 
acceptable norms of behavior. 

19. The command clearly establishes responsibility and accountability, for safe flight 
operations at all levels. 

20. Command leaders have an accurate perception of the motivation, morale, and job 
satisfaction level of their people. 

21. Command leadership reacts well, and readily adapts to unexpected changes. 

PROPOSED COMPONENT 1: PROCESS AUDITING 

1. My command adequately reviews and updates safety practices. 

2. The command has a dedicated program that targets individual training deficiencies. 

3. My command monitors maintainer qualifications. 

4. Support equipment licensing is monitored in this command. 

5. Tool control is taken seriously at my command. 

6. CDIs/QARs routinely monitor maintenance evolutions. 

7. My command uses safety staff to manage personnel at risk. 

8. The command uses medical staff to manage occupational hazards and personnel at 
risk. 
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PROPOSED COMPONENT 2: REWARD SYSTEM AND SAFETY CULTURE 



1. My command recognizes individual safety achievement through rewards and 
incentives. 

2. Unprofessional behavior is not tolerated in the maintenance department. 

3. Supervisors encourage reporting safety concerns without fear of retribution. 

4. Supervisors discourage violations of SOPs, or NAMP guidelines. 

5. My MO/MCPO understands if I feel uncomfortable performing maintenance duties 
due to personal issues. 

6. Violations of SOP, NAMP guidelines, or other procedures are common in my 
command. 

7. Peer influence discourages violations of SOP, NAMP guidelines or other procedures. 

8. Personnel are uncomfortable telling supervisors about personal problems including 
illness. 

9. Individuals feel free to report safety violations, unsafe performance, or other unsafe 
behavior. 

10. Our command climate promotes safe maintenance and flight operations. 

PROPOSED COMPONENT 3: QUALITY 

1 . My command has established standards and maintains quality control. 

2. CDIs/QARs are sought after positions in my command. 

3. Inspectors perform all required actions before sign off. 

4. To meet operational commitments, supervisors allow “cutting comers.” 

5. Maintainer staffing is sufficient from shift to shift. 

6. Proper tools and equipment are available, servicable and used. 

7. Required publications are available, current and used. 

8. Maintenance gripes are either corrected or addressed prior to flight. 

9. My command has a reputation for quality maintenance. 

10. The QA division is respected in my command. 

11. Signing off PQS/JQRs/PARs is taken seriously and not gun decked. 

12. Maintenance quality on detachments is the same as that in homeport. 

PROPOSED COMPONENT 4: RISK MANAGEMENT 

1. My command temporarily restricts maintainers who are having personal problems. 

2. Based upon my command’s current manning and assets, it is over-committed. 

3. Supervisors manage hazards associated with maintenance and flight line operations. 

4. Supervisors are more concerned with mission completion than aircraft maintenance. 

5. My division CPO is aware of individual daily workload requirements. 

6. Unsafe conditions are recognized and addressed by M/C, Q/A, or W/C supervisors. 

7. Personnel turnover negatively affects my command’s ability to operate safely. 

8. Day and night check have equal workload and are equally stressful/fatiguing. 

9. I am provided adequate resources (time, personnel, and equipment ) to accomplish my 
job. 
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10. Safety decisions are made at the proper command levels. 

1 1. Safety is part of maintenance planning, and additional trainmg/support is provided as 
needed. 

12. Maintainers are never purposely put in an unsafe situation to meet the flight schedule. 

PROPOSED COMPONENT 5: COMMAND AND CONTROL 

1. My command ensures all maintainers are responsible and accountable for safe 
maintenance. 

2. My command ensures the uniform enforcement of SOPs among unit maintenance 
personnel. 

3. Supervisors communicate command safety goals, programs, and procedures. 

4. Supervisors are actively involved in the safety program and management of safety 
matters. 

5. Supervisors set the example for compliance to established maintenance standards. 

6. Supervisors are responsive to unexpected changes and anticipate potential hazards. 

7. W/C supervisors are respected by the maintenance chief/officer. 

8. All maintenance evolutions are properly supervised by qualified personnel. 

9. Maintenance control is effective in managing all maintenance activities. 

10. Multiple job assignments and collateral duties adversely affect maintenance. 

1 1 . In my command, we believe safety is an integral part of all maintenance and flight 
line operations. 

12. Safety education and training in my command are comprehensive and effective. 

13. The safety department is respected by supervisors and maintainers. 

14. Maintenance Safety Petty Officer is a sought after billet in my command. 

PROPOSED COMPONENT 6: COMMUNICATION / RELATIONSHIPS 

1 . My command has a problem with passdown between shifts. 

2. Within my unit, good communication flow exists up and down the chain of command. 

3. Coordination is conducted between the M/C, W/C and QA prior to incorporation of 
TDs. 

4. Work center supervisors, division CPOs and M/C work well together. 

5. Aircraft moves are briefed and detailed personnel are qualified. 

6. Maintainers are briefed on potential hazards associated with maintenance activities. 

7. My supervisor shields me from outside pressures, which may affect my work. 

8. QARs are never pressured by the maintenance supervisors to sign off a gripe. 

9. Maintenance Control never troubleshoots aircraft discrepancies. 

10. QARs are viewed as helpful, and QA is not “feared” in my command. 

1 1. 1 feel I get all information (internal and external) required to perform my job safely. 



APPENDIX B. CANDIDATE MAINTENANCE QUESTIONS 

1. My command has a reputation for high quality maintenance. 

2. My command is genuinely concerned about safety. 

3. Command leadership is successful in communicating its safety goals to unit 
personnel. 

4. My command provided a positive climate that promotes safe maintenance and flight 
operations. 

5. Command leaderships is actively involved in the safety program and management of 
safety matter. 

6. Command leadership sets the example for compliance to established standards. 

7. My command conducts adequate reviews and updates of safety practices. 

8. My command ensures that all unit members are responsible and accountable for safe 
maintenance and flight operations. 

9. Command leadership permits “cutting comers” to get the job done. 

10. Safety decisions are made at the proper levels by most qualified people in my 
command. 

11. Command leadership encourages reporting safety discrepancies without fear of 
negative repercussions. 

12. Individuals in my command are willing to report information regarding safety 
violations, marginal performance, or other unsafe behavior. 

13. My command has a defined process to set training goals and to review performance. 

14. Loss of experienced personnel has negatively affected my command’s ability to 
operate safely. 

15. Command leadership willingly assists in providing advice concerning safety matters. 

16. My command has established quality standards and strives to maintain quality control. 

17. Command leadership considers safety issues during the formation and execution of 
operational and training plans. 

18. Command leadership has a clear picture of the risks associated with its maintenance 
and flight line operations. 

19. My command takes the time to identify and assess risks associated with its 
maintenance and flight line operations. 

20. My command does a good job managing risks associated with its maintenance and 
flight line operations. 

21. My command closely monitors proficiency and currency standards to ensure 
maintenance personnel are qualified to perform their assigned tasks. 

22. My command provides adequate safety backups to catch possible human error during 
hi-risk tasks. 

23. Command leadership reacts well to unexpected changes to its plans. 

24. In my command, we believe safety is an integral part of al maintenance and flight line 
operations. 

25. My command has increased the chances of a mishap due to inadequate or incorrect 
risk assessment. 
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26. My command does not hesitate to temporarily restrict from maintenance tasks 
individuals who are under high personal stress. 

27. 1 am adequately trained to safely conduct all of my assigned tasks. 

28. 1 am provided adequate resources (time, staffing, budget, and equipment) to 
accomplish my job. 

29. No equivalent. 

30. My collateral duties adversely affect my ability to accomplish my maintenance tasks. 

3 1 . Morale and motivation in my command are high. 

32. My command ensures the uniform enforcement of operating standards among unit 
personnel. 

33. 1 have adequate time to prepare for my assigned tasks. 

34. Command leadership is effective at discouraging violations of operating procedures, 
NAMP guidelines or general maintenance discipline. 

35. In my command, peer influence is effective at discouraging violations of operating 
procedures, NAMP guidelines, or general maintenance discipline. 

36. My command’s SOP is effective at promoting safe maintenance and flight line 
operations. 

37. 1 am very familiar with the policies and regulations contained in the OPNAVINST 
4790.2 series. 

38. Rest standards are enforced in my command. 

39. Based upon my command’s personnel and other assets, the command is over 
committed. 

40. In my command, PC/CDI/QAR tests and monitors are conducted as intended, to 
candidly assess maintainer qualifications. 

41. Strict enforcement of NAMP standards is upheld in my command. 

42. In my command, anyone who intentionally violates NAMP/NAMSOP is swiftly 
corrected. 

43. In my command, violations of operating procedures, NAMP regulations, or general 
maintenance or flight line operation discipline are rare. 

44. Within my command, good communication flow exists up and down the chain of 
command. 

45. My command has good two-way communication with external commands. 

46. Procedures in my command are adequate to effectively conduct Human Factors 
Councils or Boards. 

47. Human Factors councils have been successful in identifying maintenance members 
who pose a risk to safety. 

48. Human Factors Boards have been successful in managing the high-risk maintainer. 

49. My command makes effective use of the flight surgeon to help identify and manage 
high-risk personnel. 

50. My command recognizes an individual’s safety achievements through rewards and 
incentives. 

51. Safety education and training are adequate in my command. 

52. The safety department is a well-respected element of my command. 

53. The Maintenance Safety Petty Officer position is a sought after billet in my command. 
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54. My commands safety department keeps me well informed regarding important safety 
information. 

55. My command has received adequate guidance, information, and training regarding 
Maintenance Risk Management. 

56. My command has begun to implement MRM processes into decision-making at all 
levels. 

57. The QA division is well respected in my command. 

58. CDI is a sought after position in my command. 

59. QAR is a sought after position in my command. 

60. The EAWS program is considered to have integrity in my command. 

61. A coordinated brief is conducted between M/C, the W/C and QA prior to 
incorporating the AFB/AFC’s, etc. 

62. CDI/QARs perform routine monitors of maintenance evolutions. 

63. Requirements for PPE use are strictly enforced. 

64. The qualifying of CDI/QAR’s is considered to be a fair process. 

65. Decisions on who should be CDI/QAR are made fairly and without favoritism. 

66. Support equipment licensing is closely monitored. 

67. Requirements for having SE licenses are taken seriously. 

68. The “signing-off’ of PQS/JQRs/PARs, etc. is taken seriously and managed with 
integrity, (not gun decked) 

69. Maintenance Control provides for adequate time to complete the job when assigning 
tasks. 

70. Performing incomplete daily inspections is not tolerated in my command. 

71. Inspectors never sign off the “inspected by” block without physically performing the 
required inspection. 

72. Work center supervisors work well with M/C. 

73. W/C supervisors are respected by maintenance management in my command. 

74. W/C supervisors are respected by the “troops” in my command. 

75. W/C supervisors are always asked their opinion by M/C. 

76. QA is a “feared” organization in my command. 

77. All participants involved in an aircraft move are briefed prior to commencing the 
move. 

78. All maintenance evolutions are well supervised. 

79. Maintenance evolutions are OVER supervised. 

80. Anyone in my command can stop a maintenance evolution if they feel that an unsafe 
condition exists. 

81. All pilots strictly follow plane captain directions/signals. 

82. Work center shift passdowns are conducted face to face. 

83. Daily maintenance meetings encourage feedback from work centers. 

84. All work centers utilize a written passdown log. 

85. Division CPOs ensure 100% participation at monthly maintenance safety meetings. 

86. Workers with “bad attitudes” are not permitted to work on or around aircraft or other 
equipment. 

87. Utilizing someone’s NALCOMIS password is strictly prohibited. 

88. Maintenance Control is ALWAYS in control of maintenance. 
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89. 1 have confidence that all personnel are qualified to perform their assigned tasks. 

90. “Down” gripes are taken seriously by QA. 

91. “Down” gripes are taken seriously by M/C. 

92. Repeat gripes are monitored by QA and regarded as serious. 

93. Publications are always used when performing maintenance. 

94. Maintenance Control never troubleshoots from behind the M/C counter. 

95. Plane captains always carry daily decks when performing daily inpsections. 

96. A799s are taken seriously. 

97. Maintenance training is conducted regularly and taken seriously by maintenance 
management. 

98. CPO’s are regarded as the technical experts and get involved in troubleshooting tough 
gripes. 

99. CPO’s over supervise maintenance tasks. 

100. Tool Control is taken seriously at my command. 

101. The maintenance control chief is respected. 

102. My CO is proud of the maintenance department. 

103. Squadron pilots feel comfortable flying the aircraft in this command. 

104. Our maintenance department is considered to be the best in the wing. 

105. Short cuts are tolerated when the flight schedule dictates. 

106. The maintenance department works for the operations department. 

107. The goals of the maintenance department are clearly defined. 

108. Command leadership is more concerned with mission completion that aircraft 
material conditions. 

109. PPE is available for use when needed. 

1 10. Most maintenance is performed by NX. 

111. NX maintainers are considered to be proficient. 

1 12. NX maintainers are considered to be as professional as DX. 

1 13. “Khaki presence” is evident at this command. 

1 14. NX is well represented with CPO’s. 

1 15. Shortcuts are tolerated more at night than DX. 

1 16. Working nights is more risky than working days. 

1 17. Maintainers would never be put in an unsafe situation to make the flight schedule. 

1 18. Maintenance performed on detachments is a lesser quality than that at homeport. 

1 19. W/C supervisors are allowed to run their shops without interference from M/C. 

120. W/C supervisors feel pressure from M/C. 

121. Maintenance is only performed “by the book.” 

122. Work priorities are clearly set at the shift change maintenance meetings. 

123. Use of overdue calibration equipment is not tolerated. 

124. Maintenance personnel are often over worked. 

125. Personnel are allowed to work on aircraft or equipment when “hung-over.” 

126. Personnel are not comfortable telling supervisors that they have personal 
problems. 

127. NX works longer shifts than DX. 

128. Personnel are assigned more than one task at a time making it difficult to 
concentrate on one job. 
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129. My maintenance officer/MCPO would understand if I did not feel comfortable 
working on an aircraft or equipment because I was having a family problem. 

130. Publications are available, understandable, and used. 

131. Iam comfortable admitting to my supervisor that I have made a mistake. 

132. Supervisors tolerate “short-cuts.” 

133. Most maintainers would rather cover up a mistake than admit to it. 

134. Admitting to having an illness such as a cold is considered trying to “weasel out 
of work.” 

135. My work center is undermanned. 

136. My division CPO is aware of my daily workload. 

137. M/C would never put me in work on a job that could put me in an unsafe 
situation. 

138. I am never rushed to complete a task. 

139. Much of the pressure from M/C to complete a job is perceived and not real. 

140. My supervisor is responsible for informing M/C if I feel they are 
pressuring/rushing me. 

141. All known problems are addressed or corrected prior to an aircraft being released 
“safe for flight.” 

142. I am considered to be an important part of our maintenance department. 

143. I feel comfortable reporting someone who I consider puts me in an unsafe 
situation. 

144. Unprofessional behavior is not tolerated in the maintenance department. 

145. Maintenance management encourages me to perform maintenance “by the book.” 

146. Disciplinary problems are not considered by maintenance management to affect a 
person’s ability to perform his/her job. 

147. If an unsafe condition existed, it would be recognized by M/C, Q/A, or the W/C 
supervisor before it became a problem. 

148. All personnel in my department are physically capable of doing there jobs. 

149. Intentional violations of proper procedures are dealt with swiftly. 

150. QA is looking out for my safety. 

151. I am briefed on al possible hazards associated with performing a job prior to 
starting that job. 

152. QA is considered to be the technical experts. 

153. QA is never pressured by maintenance leadership to sign off a gripe. 

154. My job is as important as the aircrews. 

155. My job (role) is considered to be important in achieving the overall command 
goal. 
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APPENDIX C. PROTOTYPE MCAS 



Purpose : The purpose of this survey is to try and gain valuable insight into the maintenance community’s 
perception concerning aviation mishaps within the Navy and Marine Corps. Your participation and answers 
will be used as a guide in the Navy’s on-going efforts to lower the aviation mishap rate. 

The first fifteen questions, part I, regard biographical data; information particular to yourself. This 
information will aid in the analysis of your responses. NO attempts will be made to identify individual 
respondents or their organizations. 

Part II has 67 questions pertaining to the maintenance community. Please respond to the questions with 
the answer that most correctly reflects your honest opinion. Using a #2 pencil, completely darken each 
response. 

Thank you in advance for your participation! 

PLEASE RESPOND TO EACH ITEM. 



1 . 


Your rank? ( ) E-l - 


E-3 


t 


) E-4 - 


E-6 ( 


n 

o 


E-7 


+ ( 


) Officer 


2. 


Your community? 




















VFA ( ) HS 


( 


) 


VMFA 


( ) 


VF 


( 


) 


HSL ( ) 




VMA ( ) HC 


( 


) 


VP 


( ) 


HCS 


( 


) 


VX ( ) 




VR ( ) VQ 


( 


) 


VAQ 


( ) 


VAW 


( 


) 




3 . 


Your designator? ( LDO, 


152X, 


etc ) ? 






/ 


NEC 





4. Are you currently a department head? ( ) Yes ( ) No 

5. Your service? ( ) USN ( ) USNR TAR ( ) SELRES ( ) Other 

6. Your shift? ( ) DX ( ) NX ( ) MidX ( ) Other, specify 

7 . Total years of service? 

8. Total years of Aviation Maintenance experience? 

9. A-School graduate? ( ) Yes ( ) No ( ) N/A 

10. Education level : ( ) GED ( ) High School ( )College, # of years. 

11. Unit home location? ( ) East coast ( )West Coast ( ) Other 

12. Your rating? ( ) AD/ AM ( ) AE/AT ( ) PR/AME ( )AO ( ) Other 

13. Your age? ( ) 17-20 ( ) 21-25 ( ) 25-30 ( ) 30+ 

14. Current maintenance qualifications? 

( )Safe for Flight ( )QAR 
( ) CDI ( ) Supervisor 

( )SPO ( ) N /A 



15. Duty: ( ) Shore ( ) Sea 
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Part II 



1. My command adequately reviews and updates ( ) 

safety practices. 

2. The command has a dedicated program that ( ) 

targets individual training deficiencies. 

3. My command monitors maintainer qualifications. ( ) ( ) 

4. Support equipment licensing is monitored ( ) 

in this command. 



( ) ( ) ( ) ( ) 

( ) ( ) ( ) ( ) 

( ) ( ) ( ) 

( ) ( ) ( ) ( ) 



5. Tool Control is taken seriously at my command. 

6. My command recognizes individual safety 
achievement through rewards and incentives. 



( ) ( ) ( ) ( ) ( ) 

( ) ( ) ( ) ( ) ( ) 



7. Unprofessional behavior is not tolerated in the 
maintenance department. 

8. My command has a problem with passdown 
between shifts. 



( ) 



( ) 



( ) ( ) ( ) ( ) 



( ) ( ) ( ) ( ) 



9. My command follows established standards 
and maintains quality control. 

10. CDIs/QARs are sought after positions in my 
command. 



( ) 



( ) 



( ) ( ) ( ) ( ) 



( ) ( ) ( ) ( ) 



1 1 . My command temporarily restricts maintainers 
who are having personal problems. 

12. Based upon my command's current manning 
and assets, it is over-committed. 



( ) 



( ) 



( ) ( ) ( ) ( ) 



( ) ( ) ( ) ( ) 



13. My command ensures all maintainers are responsible 
and accountable for safe maintenance. 



( ) 



( ) ( ) ( ) ( ) 



14. My command ensures the uniform enforcement 
of SOPs among unit maintenance personnel. 

15. Within my unit, good communication flow 
exists up and down the chain of command. 

16. Coordination is conducted between M/C , W/C 
and QA prior to incorporation of TDs. 



( ) ( ) ( ) ( ) ( ) 

( ) ( ) ( ) ( ) ( ) 

( ) ( ) ( ) ( ) ( ) 



17. Inspectors perform all required actions before ( ) ( ) ( ) ( ) ( ) 

sign off. 

18. Supervisors encourage reporting safety concerns ( ) ( ) ( ) ( ) ( ) 

without fear of retribution. 
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19. Supervisors discourage violations of SOPs, 
or NAMP guidelines. 

20. To meet operational commitments, supervisors 
allow “cutting comers”. 

21. Supervisors manage the hazards associated with 
maintenance and flight line operations. 

22. Supervisors are more concerned with mission 
completion than aircraft maintenance. 

23. My division CPO is aware of individual 
daily workload requirements. 

24. Unsafe conditions are recognized and 
addressed by M/C, Q/A, or W/C supervisors. 

25. Supervisors communicate command safety goals, 
programs and procedures. 

26. Supervisors are actively involved in the safety 
program and management of safety matters. 

27. Supervisors set the example for compliance to 
established maintenance standards. 



( ) ( ) ( ) ( ) ( ) 

( ) ( ) ( ) ( ) ( ) 

( ) ( ) ( ) ( ) ( ) 

( ) ( ) ( ) ( ) ( ) 

( ) ( ) ( ) ( ) ( ) 

( ) ( ) ( ) ( ) ( ) 

( ) ( ) ( ) ( ) ( ) 
( ) ( ) ( ) ( ) ( ) 

( ) ( ) ( ) ( ) ( ) 



28. Supervisors are responsive to unexpected changes ( ) ( ) ( ) ( ) ( ) 

and anticipate potential hazards. 

29. W/C supervisors are respected by the maintenance ( ) ( ) ( ) ( ) ( ) 

chief/officer. 

30. All maintenance evolutions are properly supervised ( ) ( ) ( ) ( ) ( ) 

by qualified personnel. 

31. Work center supervisors , division CPOs and ( ) ( ) ( ) ( ) ( ) 

M/C work well together. 

32. Aircraft moves are briefed and detailed personnel ( ) ( ) ( ) ( ) ( ) 

are qualified. 

33. Maintained are briefed on potential hazards ( ) ( ) ( ) ( ) ( ) 

associated with maintenance activities. 

34. My supervisor shields me from outside ( ) ( ) ( ) ( ) ( ) 

pressures which may affect my work. 

35. QARs are never pressured by the maintenance ( ) ( ) ( ) ( ) ( ) 

supervisors to sign off a gripe. 

36. My MO/MCPO understand if I feel uncomfortable ( ) ( ) ( ) ( ) ( ) 

performing maintenance due to personal issues. 

37. Maintainer staffing is sufficient from shift to shift. ( ) ( ) ( ) ( ) ( ) 
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38. Maintenance control is effective in managing ( ) ( ) ( ) ( ) ( ) 

all maintenance activities. 

39. Multiple job assignments and collateral duties ( ) ( ) ( ) ( ) ( ) 

adversely affect maintenance. 

40. Personnel turnover negatively affects my ( ) ( ) ( ) ( ) ( ) 

command’s ability to operate safely. 

41. Violations of SOP, NAMP guidelines, or other ( ) ( ) ( ) ( ) ( ) 

procedures are common in my command. 

42. Proper tools and equipment are available, ( ) ( ) ( ) ( ) ( ) 

serviceable and used. 

43. Maintenance Control never troubleshoots ( ) ( ) ( ) ( ) ( ) 

aircraft discrepancies. 

44. Required publications are available, current, ( ) ( ) ( ) ( ) ( ) 

and used. 



45. Maintenance gripes are either corrected or ( ) ( ) ( ) ( ) ( ) 

addressed prior to flight. 

46. CDIs/QARs routinely monitor maintenance ( ) ( ) ( ) ( ) ( ) 

evolutions. 



47. My command has a reputation for quality maintenance. ( ) 

48. The QA division is respected in my command. ( ) 

49. Signing off PQS/JQRs/PARs is taken seriously ( ) 

and not gun decked. 

50. Maintenance quality on detachments is the same ( ) ( ) 

as that in homeport. 

5 1 . Day and Night Check have equal workload and ( ) 

are equally stressful/fatiguing. 

52. QARs are viewed as helpful, and QA is ( ) 

not “feared'’ in my command. 

53. Peer influence discourages violations of SOP, ( ) 

NAMP guidelines or other procedures. 

54. Personnel are uncomfortable telling supervisors ( ) 

about personal problems including illness. 

55. Iam provided adequate resources (time, personnel ( ) 

and equipment) to accomplish my job. 

56. In my command, we believe safety is an integral ( ) 

part of all maintenance and flight line operations. 
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( ) ( ) ( ) 

( ) ( ) ( ) 

( ) ( ) ( ) 

( ) ( ) ( ) 
( ) ( ) ( ) 

( ) ( ) ( ) 

( ) ( ) ( ) 

( ) ( ) ( ) 

( ) ( ) ( ) 

( ) ( ) ( ) 



( ) 
( ) 
( ) 



( ) 
( ) 
( ) 
( ) 
( ) 
( ) 



57. I feel I get all information (internal and external) ( ) ( ) ( ) ( ) ( ) 

required to perform my job safely. 

58. Individuals feel free to report safety violations, ( ) ( ) ( ) ( ) ( ) 

unsafe performance, or other unsafe behavior. 

59. My command uses safety staff to manage ( ) ( ) ( ) ( ) ( ) 

personnel at risk. 

60. Our command climate promotes safe maintenance ( ) ( ) ( ) ( ) ( ) 

and flight operations. 

61. Safety decisions are made at the proper ( ) ( ) ( ) ( ) ( ) 

command levels. 

62. Safety is part of maintenance planning, and ( ) ( ) ( ) ( ) ( ) 

additional training/support is provided as needed. 

63. Maintainers are never purposely put in an ( ) ( ) ( ) ( ) ( ) 

unsafe situation to meet the flight schedule. 

64. Safety education and training in my command are ( ) ( ) ( ) ( ) ( ) 

comprehensive and effective. 

65. The safety department is respected by ( ) ( ) ( ) ( ) ( ) 

supervisors and maintainers. 

66. Maintenance Safety Petty Officer is a sought after ( ) ( ) ( ) ( ) ( ) 

billet in my command. 

67. The command uses medical staff to manage ( ) ( ) ( ) ( ) ( ) 

occupational hazards and personnel at risk. 
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APPENDIX D. COMPONENT SURVEY DESCRIPTIVE RESULTS 



Question 


Fleet Support 
Squadron 


Patrol 

Squadron 


Combat Support 
Squadron 


Combined 

Squadrons 




Total 


Avg 


StdDev 


Total 


Avg 


StdDev 


Total 


Avg 


StdDev 


Total 


Avg 


StdDev 


1 


108 


4.45 


0.702 


93 


4.45 


0.634 


67 


4.39 


0.758 


268 


4.44 


0.692 


2 


108 


3.86 


0.942 


93 


3.90 


0.835 


67 


3.72 


1.070 


268 


3.84 


0.940 


3 


107 


4.44 


0.881 


92 


4.29 


0.749 


66 


4.27 


0.755 


265 


4.35 


0.849 


4 


106 


4.39 


0.952 


92 


4.36 


0.764 


67 


4.24 


0.836 


265 


4.34 


0.936 


5 


106 


4.54 


0.864 


93 


4.86 


0.406 


67 


4.61 


0.673 


266 


4.67 


0.803 


46 


105 


4.24 


1.020 


90 


4.18 


0.680 


65 


4.20 


0.775 


260 


4.21 


0.960 


59 


104 


3.92 


1.050 


90 


3.91 


0.802 


67 


3.73 


1.109 


261 


3.87 


1.088 


67 


105 


3.30 


1.050 


91 


3.49 


1.004 


66 


3.08 


1.154 


262 


3.31 


1.119 



MOSE Process Auditing Component 



Question 


Fleet Support 
Squadron 


Patrol 

Squadron 


Combat Support 
Squadron 


Combined 

Squadrons 




Total 


Avg 


StdDev 


Total 


Avg 


StdDe\ 


Total 


Avg 


StdDev 


Total 


Avg 


StdDev 


6 


106 


3.60 


0.976 


93 


3.83 


3.828 


67 


3.90 


1.103 


266 


3.78 


1.120 


7 


105 


3.97 


1.222 


92 


4.14 


0.859 


67 


3.87 


1.100 


264 


3.99 


1.115 


18 


107 


4.27 


0.960 


93 


4.22 


0.845 


67 


4.01 


1.007 


267 


4.17 


0.968 


19 


105 


3.40 


1.350 


92 


4.11 


0.883 


66 


4.03 


0.960 


263 


3.85 


1.216 


36 


101 


3.38 


1.120 


89 


3.28 


0.988 


64 


3.17 


1.121 


254 


3.28 


1.187 


41 


105 


2.45 


1.180 


93 


2.11 


1.098 


66 


1.94 


1.006 


264 


2.16 


1.145 


53 


105 


3.68 


1.090 


92 


3.62 


0.959 


66 


3.53 


1.099 


263 


3.61 


1.107 


54 


105 


2.64 


1.220 


91 


2.49 


1.109 


65 


2.91 


1.169 


261 


2.68 


1.202 


58 


105 


4.23 


0.990 


92 


4.35 


0.702 


66 


4.05 


1.143 


263 


4.21 


1.044 


60 


105 


4.38 


0.810 


92 


4.42 


0.683 


67 


4.28 


0.901 


264 


4.36 


0.914 



MOSE Reward System and Safety Culture Component 



Question 


Fleet Support 
Squadron 


Patrol 

Squadron 


Combat Support 
Squadron 


Combined 

Squadrons 




Total 


Avg 


StdDev 


Total 


Avg 


StdDev 


Total 


Avg 


StdDev 


Total 


Avg 


StdDev 


9 


107 


4.25 


0.814 


93 


4.30 


0.672 


66 


4.36 


0.572 


266 


4.31 


0.756 


10 


103 


3.70 


1.027 


92 


3.73 


0.800 


65 


3.75 


0.936 


260 


3.73 


1.048 


17 


106 


4.30 


0.790 


92 


4.22 


0.836 


67 


4.24 


0.854 


265 


4.25 


0.898 


20 


106 


1.99 


1.190 


92 


1.82 


1.016 


66 


2.02 


1.143 


264 


1.94 


1.128 


37 


101 


2.70 


1.160 


91 


3.16 


0.981 


67 


2.79 


1.122 


259 


2.89 


1.186 


42 


105 


3.54 


1.300 


90 


3.89 


0.965 


66 


3.89 


1.111 


261 


3.78 


1.215 


44 


107 


4.21 


0.990 


89 


4.33 


0.750 


66 


4.32 


0.914 


262 


4.28 


0.930 


45 


103 


4.32 


1.030 


91 


4.34 


0.749 


66 


4.35 


0.850 


260 


4.34 


1.063 


47 


106 


4.41 


0.880 


92 


4.51 


0.703 


66 


4.62 


0.576 


264 


4.51 


0.847 


48 


105 


4.03 


1.140 


92 


4.34 


0.745 


66 


4.29 


0.855 


263 


4.22 


1.047 


49 


103 


3.84 


1.230 


91 


4.01 


0.960 


66 


3.88 


1.060 


260 


3.91 


1.212 


50 


104 


3.64 


1.270 


92 


4.14 


0.884 


66 


4.00 


1.137 


262 


3.93 


1.219 



MOSE Quality Component 
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Question 


Fleet Support 
Squadron 


Patrol 

Squadron 


Combat Support 
Squadron 


Combined 

Squadrons 




Total 


Avg 


StdDev 


Total 


Avg 


StdDev 


Total 


Avg 


StdDev 


Total 


Avg 


StdDev 


11 


103 


3.04 


1.056 


91 


3.19 


0.918 


65 


3.28 


0.875 


259 


3.17 


1.050 


12 


105 


2.93 


1.120 


92 


3.15 


0.983 


66 


3.00 


1.123 


263 


3.03 


1.116 


21 


106 


3.94 


1.000 


92 


3.89 


0.805 


66 


3.98 


0.903 


264 


3.94 


0.969 


22 


107 


2.32 


1.290 


92 


2.03 


0.966 


66 


2.15 


1.056 


265 


2.17 


1.135 


23 


105 


3.80 


1.100 


90 


3.73 


0.981 


63 


3.65 


1.109 


258 


3.73 


1.125 


24 


107 


4.33 


0.760 


92 


4.30 


0.752 


66 


4.21 


0.814 


265 


4.28 


0.813 


40 


105 


2.92 


1.270 


91 


2.93 


0.892 


66 


2.86 


1.201 


262 


2.91 


1.165 


51 


103 


2.60 


1.220 


87 


3.21 


0.904 


66 


2.68 


1.166 


256 


2.83 


1.189 


55 


104 


3.50 


1.150 


88 


3.81 


0.981 


67 


3.51 


1.106 


259 


3.60 


1.167 


61 


106 


4.08 


1.020 


92 


4.24 


0.717 


67 


4.12 


0.930 


265 


4.15 


0.964 


62 


104 


4.15 


0.930 


92 


4.27 


0.772 


67 


4.24 


0.818 


263 


4.22 


0.986 


63 


104 


3.80 


1.190 


92 


4.07 


1.003 


66 


4.12 


0.969 


262 


3.99 


1.174 



MOSE Risk Management Component 



Question 


Fleet Support 
Squadron 


Patrol 

Squadron 


Combat Support 
Squadron 


Combined 

Squadrons 




Tot 


Avg 


SD 


Tot 


Avg 


SD 


Tot 


Avg 


SD 


Tot 


Avg 


SD 


13 


106 


4.35 


0.829 


92 


4.33 


0.730 


67 


4.28 


0.692 


265 


4.32 


0.843 


14 


106 


3.98 


0.976 


92 


4.03 


0.870 


66 


4.12 


0.775 


264 


4.04 


0.954 


25 


106 


4.08 


0.950 


92 


4.12 


0.782 


66 


3.98 


0.920 


264 


4.06 


0.951 


26 


106 


3.98 


1.000 


92 


4.18 


0.662 


66 


4.05 


0.919 


264 


4.07 


0.939 


27 


107 


3.95 


0.960 


91 


4.14 


0.754 


66 


3.97 


0.911 


264 


4.02 


0.913 


28 


106 


3.97 


0.950 


92 


4.01 


0.791 


66 


3.88 


0.851 


264 


3.95 


0.933 


29 


107 


3.79 


1.070 


91 


3.45 


1.213 


66 


3.83 


1.090 


264 


3.69 


1.155 


30 


106 


3.85 


1.090 


92 


4.15 


0.725 


66 


4.08 


0.966 


264 


4.03 


1.011 


38 


105 


3.92 


1.090 


93 


3.82 


0.988 


66 


3.77 


1.049 


264 


3.84 


1.114 


39 


104 


3.30 


1.200 


92 


3.10 


0.973 


66 


3.36 


1.159 


262 


3.25 


1.175 


56 


106 


4.49 


0.810 


92 


4.57 


0.599 


67 


4.61 


0.602 


265 


4.56 


0.792 


64 


105 


4.07 


0.960 


92 


4.10 


0.757 


67 


4.12 


0.879 


264 


4.09 


0.968 


65 


104 


3.92 


1.100 


92 


4.07 


0.823 


67 


4.01 


1.037 


263 


4.00 


1.100 


66 


102 


3.03 


1.100 


90 


3.21 


0.966 


67 


3.10 


1.032 


259 


3.12 


1.126 



MOSE Command and Control Component 



Question 


Fleet Support 
Squadron 


Patrol 

Squadron 


Combat Support 
Squadron 


Combined 

Squadrons 




Total 


Avg 


StdDev 


Total 


Avg 


StdDev 


Total 


Avg 


StdDev 


Total 


Avg 


StdDev 


8 


102 


2.78 


0.953 


92 


2.38 


1.137 


67 


2.49 


1.146 


261 


2.55 


1.237 


15 


107 


3.75 


1.222 


93 


3.51 


1.039 


67 


3.22 


1.152 


267 


3.49 


1.175 


16 


103 


3.88 


0.953 


91 


3.65 


0.970 


67 


3.73 


0.827 


261 


3.75 


1.055 


31 


106 


3.78 


1.100 


92 


3.60 


1.070 


66 


3.61 


1.149 


264 


3.66 


1.141 


32 


105 


4.00 


1.050 


92 


4.40 


0.696 


66 


3.91 


1.092 


263 


4.10 


1.060 


33 


104 


3.91 


0.980 


91 


4.07 


0.892 


65 


4.00 


0.952 


260 


3.99 


1.053 


34 


103 


2.91 


1.010 


87 


3.21 


1.013 


64 


2.91 


1.019 


254 


3.01 


1.092 


35 


103 


3.66 


1.230 


91 


3.53 


1.036 


66 


3.45 


1.126 


260 


3.55 


1.228 


43 


103 


2.70 


1.210 


90 


2.41 


1.189 


66 


2.61 


1.175 


259 


2.57 


1.236 


52 


106 


3.89 


1.080 


90 


4.08 


0.768 


66 


3.94 


0.943 


262 


3.97 


1.006 


57 


105 


3.96 


1.050 


91 


4.13 


0.748 


67 


4.12 


1.023 


263 


4.07 


1.035 



MOSE Communication / Relationship Component 
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APPENDIX E. SUGGESTED QUESTIONS MODIFICATIONS 



9. My command adequately reviews and updates safety practices. 

1 . My command follows established standards and maintains quality control. 

13. My command ensures all maintainers are responsible and accountable for safe 
maintenance. 

New question: My command adequately reviews and updates safety practices, follows 
established standards and maintains quality control, while ensuring that all maintainers 
are responsible and accountable for safe maintenance. 

3. My command monitors maintainer qualifications. 

4. Support equipment licensing is monitored in this command. 

New question: My command monitors maintainer qualifications and support equipment 
licensing. 

57. I feel I get all information (internal and external) required to perform my job safely. 

58. Individuals feel free to report safety violations, unsafe performance, or other unsafe 
behavior. 

New question: I feel I get all information required to perform my job safely, and feel free 
to report safety violations, unsafe performance or other unsafe behavior. 

30. All maintenance evolutions are properly supervised by qualified personnel. 

33. Maintainers are briefed on potential hazards associated with maintenance activitites 

New Question: Qualified personnel properly supervise all maintenance evolutions and 
maintainers are briefed on the potential hazards associated with maintenance activities. 

42. Proper tools and equipment are available, serviceable and used. 

55. I am provided adequate resources (time, personnel, and equipment) to accomplish my 
job. 

New question: Proper tools and equipment are available, serviceable and used an I am 
provided adequate resources (time, personnel) to accomplish my job. 

49. Signing off PQS/JQRs/PARs is taken seriously and not gun decked. 

50. Maintenance quality on detachments is the same as that in homeport. 

New Question: Signing off PQS/JQRs/PARs is taken seriously, not gun decked and 
maintenance quality is as high on detachments as it is in homeport. 
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2. The command has a dedicated program that targets individual training deficiencies. 
14. My command ensures the uniform enforcement of SOPs among maintenance 
personnel. 

New question: My command has a dedicated program that targets individual training 
deficiencies and ensures the uniform enforcement of SOPs among maintenance 
personnel. 

37. Maintainer staffing is sufficient from shift to shift. 

51. Day and night check have equal workload and are equally stressful / fatiguing. 

New question: Maintainer staffing is sufficient, is equally worked and is equally stressed 
/ fatigued from shift to shift. 

16. Coordination is conducted between M/C, W/C, and QA prior to incorporation of TDs. 

38. Maintenance control is effective in managing all maintenance activities. 

New question: Maintenance control is effective in managing all maintenance activities, 
coordinating between M/C, W/C, and QA prior to the incorporations of TDs. 

20. To meet operational commitments, supervisors allow cutting comers. 

22. Supervisors are more concerned with mission completion than aircraft maintenance. 

New question: Supervisors are more concerned with proper aircraft maintenance than 
mission completion and do not allow cutting comers to meet operational commitments. 

64 Safety education and training in my command are comprehensive and effective. 

65. The safety department is respected by supervisors and maintainers. 

New question: Safety education and training in my command are comprehensive and 
effective and the safety department is respected by the supervisors and maintainers. 



52. QARs are viewed as helpful, and QA is not “feared” in my command. 

56. In my command, we believe safety is an integral part of all maintenance and flight 
line operations. 

New question: Safety is an integral part of this command’s maintenance planning/flight 
line operations, where QARs are helpful and the QA division is not “feared”. 

19. Supervisors discourage violations of SOPs, or NAMP guidelines. 

53. Peer influence discourages violations of SOP, NAMP guidelines or other procedures. 

New question: Violations of SOP, NAMP guidelines or other procedures are discouraged 
in this command. 
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5. Tool control is taken seriously in my command. 

47. My command has a reputation for quality maintenance. 

New question: My command has a reputation for quality maintenance and tool control is 
taken seriously. 

18. Supervisors encourage reporting safety concerns without fear of retribution. 

21. Supervisors manage the hazards associated with maintenance and flight line 
operations. 

24. Unsafe conditions are recognized and addressed by M/C, Q/A, or W/C supervisors. 

New question: Safety concerns or unsafe hazards associated with maintenance and flight 
line operations can be reported without a fear of retribution knowing that the W/C, Q/A, 
or M/C supervisors will address and manage them for proper corrections. 

1 1 . My command temporarily restricts maintainers who are having personal problems. 
36. My MO/MCPO understand if I feel uncomfortable performing maintenance due to 
personal issues 

67. The command uses medical staff to manage occupational hazards and personnel at 
risk. 

New question: Medical staff is used to help identify, manage, and temporarily restrict 
personnel with personal issues and those who pose a risk to safe maintenance in this 
command. 

10. CDIs and QARs are sought after positions in my command. 

66. Maintenance Safety Petty Officer is a sought after billet in my command. 

New question: QARs/ CDIs and Maintenance Safety Petty Officer are sought after 

billets in my command. 

45. Maintenance gripes are either corrected or addressed prior to flight. 

46. CDIs/QARs routinely monitor maintenance evolutions. 

48. The QA division is respected in my command. 

New question: The QA division is respected and CDIs / QARs routinely monitor 

maintenance evolutions ensuring that maintenance gripes are either corrected or 
addressed prior to flight. 

Question 8: 

Current: My command has a problem with passdown between shifts. 

New question: My command does not have a problem with passdown between shifts. 
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Question 12: 



Current: Based upon my command’s current manning and assets, it is over-committed. 
New question: Based upon my command’s current manning and assets, it is not over- 
committed. 

Question 20 

Current: To meet operational commitments, supervisors allow “cutting comers.” 

New question: Supervisors do not allow “cutting comers” to meet operational 
commitments. 

Question 22 

Current: Supervisors are more concerned with mission completion than aircraft 
maintenance. 

New question: Supervisors are more concerned with proper aircraft maintenance than 
mission completion. 

Question 39 

Current: Multiple job assignments and collateral duties adversely affect maintenance. 

New question: Multiple job assignments and collateral duties do not adversely affect 
maintenance. 

Question 40 

Current: Personnel turnover negatively affects my command’s ability to operate safely. 
New question: Personnel turnover does not affect my command’s ability to operate 
safely. 



Question 41 

Current: Violations of SOP, NAMP guidelines, or procedures are common in my 
command. 

New question: Violations of SOP, NAMP guidelines, or procedures are not common in 
my command. 
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Question 54 



Current: Personnel are uncomfortable telling supervisors about personal problems 
including illness. 

New question: Personnel are comfortable telling supervisors about personal problems 
including illness. 
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APPENDIX F. FINALIZED MCAS 



Purpose : The purpose of this survey is to try and gain valuable insight into the maintenance community’s 
perception concerning aviation mishaps within the Navy and Marine Corps. Your participation and answers 
will be used as a guide in the Navy’s on-going efforts to lower the aviation mishap rate. 

The first fifteen questions, part I, regard biographical data; information particular to yourself. This 
information will aid in the analysis of your responses. NO attempts will be made to identify individual 
respondents or their organizations. 

Part II has 35 questions pertaining to the maintenance community. Please respond to the questions with 
the answer that most correctly reflects your honest opinion. Using a #2 pencil, completely darken each 
response. 

Thank you in advance for your participation! 

PLEASE RESPOND TO EACH ITEM. 



1. 


Your 


rank? ( ) 


E-l - 


E-3 ( 


) E-4 - 


E-6 ( 


) CPO 


E-7 


+ ( 


) Officer 


2. 


Your 


community? 






















VFA ( ) 


HS 


( ) 


VMFA 


( ) 


VF 


( 


) 


HSL ( ) 






VMA ( ) 


HC 


( ) 


VP 


( ) 


HCS 


( 


) 


VX ( ) 






VR ( ) 


VQ 


( ) 


VAQ 


( ) 


VAW 


( 


) 




3 . 


Your 


designator? 


( LDO, 


152X, 


etc ) 


o 




/ 


NEC 





4. Are you currently a department head? ( ) Yes ( ) No 

5. Your service? ( ) USN ( ) USNR TAR ( ) SELRES ( ) Other 

6. Your shift? ( ) DX ( ) NX ( ) MidX ( ) Other 

7. Total years of service? 

8. Total years of Aviation Maintenance experience? 

9. A-School graduate? ( ) Yes ( ) No ( ) N/A 



11. Education level : ( ) GED ( ) High School ( ) College, #yrs. 

12. Unit home location? ( ) East coast ( )West Coast ( ) Other 



12. Your rating? ( ) AD/AM ( ) AE/AT ( ) PR/AME ( ) AO ( ) Other 



13 . 
14. 



Your age? ( ) 17-20 ( ) 21-25 

Current maintenance qualifications? 


( ) 


25-30 ( ) 


30 + 


( 


) Safe 


for Flight 


( ) QAR 


( 


) CDI 




( ) Supervisor 


( 


) SPO 




( ) N /A 



15. Duty: ( ) Shore ( ) Sea 
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Part II 



1. My command has a dedicated program that ( ) ( ) ( ) ( ) ( ) 

targets individual training deficiencies and ensures 

the uniform enforcement of SOPs among maintenance 
personnel. 

2. My command monitors maintainer qualifications ( ) ( ) ( ) ( ) ( ) 

and support equipment licensing. 

3. My command has a reputation for quality ( ) ( ) ( ) ( ) ( ) 

maintenance and tool control is taken seriously. 

4. Unprofessional behavior is not tolerated in the ( ) ( ) ( ) ( ) ( ) 

maintenance department. 

5. My command has a problem with passdown ( ) ( ) ( ) ( ) ( ) 

between shifts. 

6. My command adequately reviews and updates ( ) ( ) ( ) ( ) ( ) 

safety practices, follows established standards and 

maintains quality control, ensuring that all maintainers 
are responsible and accountable for safe maintenance. 

7. QARs/ CDIs and Maintenance Safety Petty Officer are ( ) ( ) ( ) ( ) ( ) 

sought after billets in my command. 

8 . Medical and safety staff are used to help identify, ( ) ( ) ( ) ( ) ( ) 

manage, and temporarily restrict personnel with personal 
issues and those who pose a risk to safe maintenance in 
this command. 

9. Based upon my command’s current manning ( ) ( ) ( ) ( ) ( ) 

and assets, it is not over-committed. 

10. Within my unit, good communication flow ( ) ( ) ( ) ( ) ( ) 

exists up and down the chain of command. 

11. Maintenance control is effective in managing all ( ) ( ) ( ) ( ) ( ) 

maintenance activities, coordinating between M/C, 

W/C, and QA prior to the incorporations of TDs. 

12. Inspectors perform all required actions before ( ) ( ) ( ) ( ) ( ) 

sign off. 

13. Safety concerns or unsafe hazards associated ( ) ( ) ( ) ( ) ( ) 

with maintenance/flight line operations can be 

reported without fear of retribution knowing that the 
W/C, Q/A, or M/C supervisors will address and manage 
them for proper corrections. 

14. Violations of SOP, NAMP guidelines or other ( ) ( ) ( ) ( ) ( ) 

procedures are discouraged in this command. 
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15. Supervisors are more concerned with proper aircraft ( ) ( ) ( ) ( ) ( ) 

maintenance than mission completion and do not 

allow cutting corners to meet operational commitments. 

16. My supervisors are aware of individual ( ) ( ) ( ) ( ) ( ) 

daily workload requirements and recognize safety 

achievements through rewards and incentives. 

17. Supervisors communicate command safety goals, ( ) ( ) ( ) ( ) ( ) 

programs and procedures. 

18. W/C supervisors are respected by the maintenance ( ) ( ) ( ) ( ) ( ) 

chief/officer. 

19. Qualified personnel properly supervise all ( ) ( ) ( ) ( ) ( ) 

maintenance evolutions and maintainers are 
briefed on the potential hazards associated with 
maintenance activities. 

20. My supervisor shields me from outside ( ) 

pressures which may affect my work. 

21. QARs are never pressured by the maintenance ( ) 

supervisors to sign off a gripe. 

22. Maintainer staffing is sufficient, is equally worked and ( ) 
is equally stressed / fatigued from shift to shift. 

23. Multiple job assignments and collateral duties ( ) 

do not adversely affect maintenance. 

24. Personnel turnover does not affect my ( ) 

command’s ability to operate safely. 

25. Violations of SOP, NAMP guidelines, or other ( ) 

procedures are not common in my command. 

26. Proper tools and equipment are available, ( ) 

serviceable and used and I am provided adequate 
resources (time, personnel) to accomplish my job. 

27. Maintenance Control never troubleshoots ( ) 

aircraft discrepancies. 

28. Required publications are available, current, ( ) 

and used. 

29. The QA division is respected and CDIs / QARs ( ) ( ) ( ) ( ) ( ) 

routinely monitor maintenance evolutions ensuring 

that maintenance gripes are either corrected or 
addressed prior to flight. 

30. Signing off PQS/JQRs/PARs is taken seriously, ( ) ( ) ( ) ( ) ( ) 

not gun decked and maintenance quality is as high 

on detachments as it is in homeport. 



( ) ( ) ( ) ( ) 

( ) ( ) ( ) ( ) 

( ) ( ) ( ) ( ) 

( ) ( ) ( ) ( ) 

( ) ( ) ( ) ( ) 

( ) ( ) ( ) ( ) 

( ) ( ) ( ) ( ) 

( ) ( ) ( ) ( ) 

( ) ( ) ( ) ( ) 
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31. Safety is an integral part of this command’s ( ) ( ) ( ) ( ) ( ) 

maintenance planning/flight line operations, where 

QARs are helpful and the QA division is not “feared”. 

32. Personnel are comfortable telling supervisors ( ) ( ) ( ) ( > ( ) 

about personal problems including illness. 

33. I feel I get all information (internal and external) ( ) ( ) ( ) ( ) ( ) 

required to perform my job safely, and feel free to 

report safety violations, unsafe performance or other 
unsafe behavior. 

34. Maintainers are never purposely put in an ( ) ( ) ( ) ( ) ( ) 

unsafe situation to meet the flight schedule. 

35 Safety education and training in my command are ( ) ( ) ( ) ( ) ( ) 

comprehensive and effective and the safety department 
is respected by the supervisors and maintainers. 
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